Jeff Galper

The last few weeks have been a whirlwind of activity that abruptly ended yesterday afternoon.

In November I was informed that Amazon would be conducting interviews with the CreateSpace interns graduating in June.  I gave HR my application and waited to be contacted about a phone interview.  Weeks passed and I eventually forgot about it – June was several months off and I wouldn’t be starting until then if I got a job.

In late December/early January the two most senior people on my team announced they’d be leaving.  One was moving to another team, the other leaving the company.  The silver lining of this news was the new positions opening up.  Since I’m still in college I was informed I didn’t qualify for the SDE position, but the support engineer on our team did.  It looked like our SE would become an SDE and I would become an SE.

After delays due to vacations and interviews the SE was offered the SDE position.  He accepted and the SE position opened up.  I put my application in and shortly after (2/17?), was offered the job.  Right around this time Amazon contacted me again about interviewing in Seattle.  I took them up on their offer and scheduled an interview for March 1st.  A few days before my Seattle trip, CreateSpace surprised me with an interview loop – for an SDE position.  I had several interviews on Wednesday (2/24) and an Amazon bar raiser interview on Friday.

I flew to Seattle the day before my Amazon interview and was picked up at the airport by my cousin Deborah and her son Cody.  Cody and I had a good time wrestling around the house while I was there.  My five hour long interview session went well and I had a good time in Seattle.

When I got back to Scotts Valley on Tuesday (3/2), the Amazon recruiter scheduled an interview with Lab126 in San Jose for that Friday.  Wednesday was mostly uneventful.  Thursday however, my SE offer was put to shame when I was offered an SDE position at CreateSpace.  I was ecstatic.

I went ahead with my Lab126 interview since I had made the commitment.  From my perspective the Lab126 interview loop was more difficult than the Amazon loop.  By this time I had pretty much decided to accept the CreateSpace offer, but was still hoping to hear back from Amazon and Lab126 first.

On Wednesday (3/10) CreateSpace revised the offer while Lab126 invited me back for another around of interviews.  A few hours later I accepted CreateSpace’s offer.  I’m now officially an SDE at CreateSpace.

Due to my endless frustrations with the Tar Library we use at work and some limitations of the format in general, I decided to create my own file format and library. There is now Feather. The Feather format can be described as a simple, extensible storage format. By using optional fields, a feather file can include all the information normally stored in a tar, or as little as the name, size, and data of a file.

While UStar formatted Tar files have a maximum filename length of 255 (with some restrictions), Feather has a maximum filename length of 23,767.  Without the GNU extensions, Tar has a maximum file size limit of 8 GB.  Feather, on the other hand, has a maximum file size of 8 EB, or 8 GB * 1024³.  Finally, Feather has native support for UTF-16, which Tar lacks completely.

The current state of the Feather repository is a bit of a mess, because it was originally programmed solely using ByteBuffers and FileChannels, but a need arose for Input and Output Streams.  I will likely remove the use of ByteBuffers and FileChannels so that there is only one official implementation of Feather.

Flicasa is my pet project (created for Django Dash) that I haven’t worked on in a few weeks.  In the meantime, however, I’ve learned quite a bit about Django and Python by convincing my boss that our new internal website should ditch Struts for something sexier.  I hope to find time to work on Flicasa soon.

Anyways, I’ve decided to make the site public.  Head to http://flicasa.com to take it for a spin.  All the public pages should work correctly, even if they’re a little unintuitive.  If you notice a problem head to the contact page and shoot me an email.

Some interesting links to help when setting up AWStats on Ubuntu:

• Installation Guide
• Turn off logging selectively

For example, my server doesn’t log any requests to my blog’s admin pages. This means the stats don’t contain hits for any work I’m doing on the site.

Mongrels used way too much ram. My little underpowered VPS couldn’t handle running a cluster of mongrels in addition to everything else I’ve got going on. Enter mod_rails and Ruby Enterprise Edition.  These two free products of Phusion provide a nice, easy to setup alternative to standard Ruby and Mongrel.  I was able to remove Mongrel from my startup and use an Apache2 module for running RoR.  The only hickup I ran into was needing to reinstall a specific version of rails.  The Ruby Enterprise Edition installed seemed to pick up the newest version (2.3.2), but apparently I had a dependency on 2.2.2.  Once I got that straightened out I was able to reclaim precious megabytes of RAM.  I once again have enough breathing room that I can focus on something other than optimizations.

I’ve nearly completed the setup of my new host.  I’ve managed to set up the following services/programs:

• Nginx as a reverse proxy
• Apache2 for heavy duty needs
• Django
• Php
• Postgresql (to use when possible)
• MySQL (to use when necessary — WordPress)
• Mercurial
• Ruby on Rails / Mongrel Cluster

It’s been a long journey and I’ve used several resources.  I’ll briefly list them for anybody else working on a similar project.

• Django, Nginx, Apache2, Postgresql — This was the foundation of my setup.  I even wrote a script to automate most of the process for any future site I create.
• Several Redmine guides and a page on Postgresql’s pg_hba.conf that helped me out with a rake problem.  A few notes about setting up Redmine: you need to install libopenssl-ruby, and the pg_hba.conf file needs an entry for 127.0.0.1/32.  The latter wouldn’t normally be an issue, but the first guide I linked recommends commenting that out.
• A page on setting up Redmine (or any mongrel cluster) with Nginx.
• The Nginx wiki.
• and several others I’ve forgotten…

Finally, I had a problem with mongrel_cluster not starting automatically.  This turned out to be due to PATH issues with the rubygem binaries.  I fixed this with this command:

for file in 'ls'
do
sudo ln -s `pwd`/$file /bin/
done

That will create a symbolic link from /bin to each file in the current directory.  For rubygems this would need to be run in

/var/lib/gems/1.8/bin

And finally, everything is working!

I finally have a real host for my website and I once again have an up to date WordPress installation.  I’ve spent the last two days installing, configuring, and securing everything.  Let’s see how this goes.