Sunday, March 28, 2010

Convert your mail.app notes to text files for use with Notational Velocity and sync

Recently I wanted to move my notes from mail.app and the notes app on the iPhone to text files. This would then allow me to use Notational Velocity with markdown hack and simpletext.ws that I use with the excellent Taskpaper iphone app by Hogg Bay Software.

You can download the script files here or copy and paste from the article.

Here’s how I got it to work. I really hope I’m not missing a trivially easy way to do this but this way did work for me!

  1. Export from Mail to an archive. This is a plain text file.
  2. Download MacVim
  3. Run a script over archive file to reduce it to text
  4. Run ruby script in new folder to convert notes to text files

Export from Mail.app to an archive

  1. Click on notes
  2. Mailbox | Archive Mailbox
  3. Choose a new empty folder like ~/Downloads/Notes
  4. Copy the Notes.mbox/mbox file to ~/Downloads/Notes. You won’t need the table_of_contents file.

The ~/Downloads/Notes/mbox file contains all the notes in a single text file with a bunch of markup.

Download MacVim

  1. Download MacVim
  2. Drag to applications folder and open

Run vim script to convert mbox text file

The script:

   :%s/ / /g
:%s/<br class="[^"]*">//g
:%s/<br>//g
:%s/=\n/ /g
:%s/<div>\([^<]*\)<\/div>/\r\1/g
:%s/^X-Universally.*\n//g
:%s/^Content-Type: text\/html;\n//g
:%s/^\tcharset=us-ascii\n//g
:%s/^X-Uniform-Type.*\n//g
:%s/^Message-Id: .*\n//g
:%s/^X-Mail-Created-Date.*\n//g
:%s/^X-Mail-Generated.*\n//g
:%s/^Content-Transfer-Encoding.*\n//g
:%s/^Mime-Version:.*\n//g
:%s/^X-Apple-Base-Url:.*//g
:%s/^X-Apple-Mail-Remote-Attachments:.*//g
:%s/^From: .*\n//g
:%s/ -0400$//g
:%s/<span [^>]*>//g
:%s/<\/span>//g
:%s/=?MACINTOSH?Q?//g
:%s/=CA? Date: .*//g
:%s/^\tcharset=.*\n//g
:%s/<div>\([^<]*\)<\/div>/\r\1/g
:%s/<div>//g
:%s/<\/div>//g
:%s/<[^>]*>//g
:%s/=CA//g
:%s/? //g
:%s/_//g
:%s/=92/'/g
:%s/\s\+$//g
:%s/#\(\a\+\)/@\1/g
:%s/^Subject: /!!!\rSubject: /g
  1. Save the above vim script to a file, like ~/notes.vim using a text editor.
  2. Right-click on the plain text mail archive file you created above (~/Downloads/Notes/mbox) and choose: Open With | MacVim. You can also launch MacVim and open the mbox file with File | Open.
  3. Type:

    :so ~/notes.vim

    Or whatever you named the vim script.

  4. This will convert the file to text. You may need to do some small other adjustments - this script worked well on my file.
    What is remaining is a list of Subject: *, Date: * and message body blocks with the !!! separating the entries. We’ll use this in the ruby script to convert them to text files.
  5. Choose File | SaveAs to save the modified file to the filename “notes.text”, let’s say ~/Downloads/Notes/notes.text

Run ruby script to convert to text file

Most mac systems these days have ruby. To test, open the Terminal and type:

    ruby --version

It should return something like:

    $ ruby --version
ruby 1.8.7 (2008-08-11 patchlevel 72) [universal-darwin10.0]

Save the script below to ~/Downloads/Notes/n.rb

    class WriteNotes
@counter = 0
def WriteNotes.writeout(subject, note, date)
# write out new file
if subject != ""
# make subject the filename
filename = subject
# limit file names to 50 chars - make longer if required
if filename.length > 50
filename = filename[0..50]
end
# strip out filename-unfriendly characters
filename = filename.strip.gsub(/ /,"_")
filename = filename.gsub(/[?:,@="'.\\\/]/,"")
# check if we have a date in the date field (basic check)
d = date[0..1].match(/[0-9]/)
# default a current date if date not found
if !d
date = Time.now.localtime.strftime("%Y-%m-%d")
end
if date.strip.length == 0
date = Time.now.localtime.strftime("%Y-%m-%d")
end
# remove comma from date if it's non-standard, like "Aug, 2"
date = date.gsub(/,/,"")
# filename template - change to taste
filename = "zan-" + date + "-" + filename + ".txt"
# write out file
notefile = File.open(filename, "w")
notefile.puts note
notefile.close
@counter = @counter + 1
# puts "__filename[#{@counter}]: " + filename
puts "Subject: [#{@counter}]" + subject
# puts "__note: " + note
end
end # writeout
subject = ""
note = ""
date = ""
file = File.open("notes.text", "r")
while (line = file.gets)
if line[0..2] == "!!!" # at a new note
# write out new text file
writeout(subject, note, date)
# init vars
subject = ""
note = ""
date = ""
else
# collect data into subject, note, date fields
m = line[0..7]
# puts m
if m == "Subject:"
subject = line[8..line.length].strip
note = ""
else
if line.strip[0..3] == "Date"
date = line.strip[6..15]
date = date.gsub(/:/,"")
date = date.gsub(/ /,"_")
else
note = note + line
end
end
end
end # while
writeout(subject, note, date)
puts "total files: #{@counter}"
file.close
rescue => err
puts "Exception: #{err}"
err
end

Open terminal (under Utilities in Applications folder) and type:

    cd ~/Downloads/Notes

Now we can run the script, type:

    ruby n.rb

If the script runs correctly, the notes.text file will be processed and each file will be written out in the format:

    zan-YYYY-MM-DD-.txt

If there is a problem, just remove the text files generated and start again (rm *.txt)

Good luck! The vim script and ruby script may require a little tweaking. Please leave a note if you found this useful.

No comments: