Delete unwanted JPG files

How to select an huge amount of photos in CR2 (canon RAW format) and JPEG?

After any photo shooting session I have always the same problem: select the right photos and delete the bad ones among hundreds clicks.
Normally I review the RAW files, then I select which photos need some post-processing actions and delete the wrong photos. After this first step, I need to delete the JPEG files corresponding to deleted CR2 raw photos.
I wrote this simple python script that searches for missing CR2 files and add a suffix “_DELETEME” to the corresponding JPG photo, in order to easily identifying the photos to delete.

Disclaimer: this is my FIRST working python script

Prerequisite: shoot in CR and JPG format using the same file name

#!/usr/bin/python
import glob
import os.path
for image in glob.glob("*.JPG"):
   fileName = os.path.splitext(image)[0]
   print "Checking " + fileName
   if (os.path.exists(fileName+".CR2") == False):
      print "Mark as DELETEME " + image
      os.rename(image, image+"_DELETEME")

print "Done! Review the DELETEME files and if it's all OK...delete them!"

Cloud backup using CrashPlan

My photo archive on my MacBook is huge and my worst nightmare is loosing all my photo.
Yes, I have a time machine drive configured on my Nas.
Yes, I have also a rsync folder on my Nas.
But I want also an online solution in case of earthquake….fire…alien invasion…and so on…
Yes…I’m paranoid.

My first choice was Google drive with the 1TB storage plan (9$ / month). It was perfect: a reliable service with an acceptable upload speed and feature set. It has only a problem: no external drive support.
My photo library is growing very fast and my internal 500gb SSD is becoming too small. So I decided to move all my library on an external drive…but with the official Google Drive client I can’t sync my library anymore.

After a little search I found Insync (15$ one time). A third part Google Client with external drive support. So beautiful!!! I started the free trial and I started to upload my library (about 100gb and 30k+ files). The first bad news was that upload speed was very….very….very slow. It took about 140 hours (5+ days) to upload all files using a 100mbit Internet connection….argh!!!
The second and worst bad news was that after some days the folder simply disappears from Google drive!!! 140 hours and then … no file on the cloud.
Insync support told me that the upload speed issue will be fixed in next release…but the lost folder was a signal: Insync is not reliable…

At the end my needs are simple:
A cloud backup solution (with unlimited space?). Mac, windows and mobile client. External drive support. Not so expensive.

My first try was with Just Cloud. In one word: s**t. The unlimited plan is not truly unlimited. No automatic video backup. No automatic backup for file larger than 1gb. A lot of restriction. No such cheaper (after the first year it will cost about 150 euros every years). After some complains with support service about the not-really-unlimited plan they give me a full money refund. Kudos to customer service. But I’ll never suggest JustCloud as a solution.

My last chance is a choice between two similar services: Backblaze and Crashplan.
Both have unlimited plan and external drive support, the monthly fee is similar ( Backblaze 50$ for 1 year – Crashplan 59.99$ for 1 year ), they have multi platform and mobile client support.

So I started the free trials (30 days for Crashplan, 15 days for Backblaze) and these are some thoughts about my experience (on internet you can find a lot of more exaustive comparison):

  • Backblaze is simpler to configure but you cannot select a single folder to backup. The Italian translation is horrible. The upload speed is faster than Crashplan but the restore options have only the “download zip” choice (100gb???) or shipped hard disk drive. No tablet (iPad and android tablet) support.
  • Crashplan has a lot of possible customization via application settings. You can select any folder to backup easily. The Mac OS X client has a good Italian translation and a better UI. The restore action can be done for any folder or file using the client, file by file. Full tablet support (and also Linux client).

It has only a great drawback: the upload speed…. I notice that it use only few bandwith: at home with a 8mbit connection the upload speed was about 400kbps. At office with a 100mbit connection the top upload speed is about 2mbps.

I tried to configure the CPU usage and upload buffer in order to speed up the upload but nothing changes: seems that this is a known issue and it is very annoying especially with the first huge backup of data. There is also the possibility to receive from Crashplan an external drive, do the initial backup and send back the drive with all your data, but for my needs I can wait some days while the client finish the upload.

Finally I found the solution and now my upload speed is about 30-40mbps (UPDATE: today is 70mbps!).

  1. First of all, for the first backup disable encryption, compression and data de-duplication from the advanced backup settings (from CrashPlan support site)
  2. Then use this link in order tu speed-up the backup
  3. If you have again speed issue try to change CrashPlan destination server using this guide and repeat until your speed grows. It works for me after 4 or 5 attempts

Finally my backup is now completed. 150GB – 40k files uploaded in about 16 hours (2 working day on a 100mbit internet connection).

I send also a mail to Code42 support team regarding my speed issue: this is its reply

Thank you for contacting Code42 support. We are a shared bandwidth service, so the speeds are not going to match your ISP’s advertised speeds. That being said, the changes you reference (http://networkrockstar.ca/2013/09/speeding-up-crashplan-backups/ and http://lifehacker.com/speed-up-crashplan-backups-and-free-up-cpu-power-with-t-1658888517) are unsupported changes to the application XML, and we highly discourage making these changes. I would suggest taking a look here:

http://support.code42.com/CrashPlan/Latest/Configuring/Unsupported_Changes_To_CrashPlan_De-Duplication_Settings

This will give you an idea and what exactly is being effected by this change, not including any other unknown consequences. Furthermore, we can’t really troubleshoot unsupported changes to the application, so I would first suggest removing the current installation and reinstalling it with a supported version, and if you continue to have issues let us know.

http://support.code42.com/CrashPlan/Latest/Troubleshooting/Uninstalling_And_Reinstalling_CrashPlan

Keep in mind that our minimum threshold is around 1Mbps, and while most users experience speeds well above this, others see speeds only marginally above this. My point is that we cannot guarantee your speeds will improve, but we can at least then look to make sure nothing is going wrong.

By the way my configuration now works fine, so I will not change it and let’s see if something goes wrong. Otherwise I will revert my config, but with the initial backup done and a lot of time saved.

Only two thing are missing: a “photo gallery” feature for mobile client and web access, and folder sharing option (two features available with Google Drive). But this is a cloud backup solution, and it simply does a cloud backup. I hope that in the next releases something will be implemented 🙂