Category Archives: Code

How I became best-in-the-world at accessing SalesForce from Elixir (and maybe Erlang too)

Over the last several months, I have been playing with Elixir.
* Reading Programming Elixir
* Reading Elixir in Action
* Following Chris McCord’s excellent Elixir tutorial
* Watching the videos from Elixir Conf
* Writing a simple IRC bot for the Puppet Labs interns
* Demo’d mesh networking, supervision trees, and hot code reloading with the IRC bot as a vehicle
* Using a hackday to write and then rewrite a service w/ Ruby/Sinatra, JRuby/TorqueBox, and Elixir/Plug with Elixir resulting is 55X the Ruby/Sinatra throughput and 2.3X the JRuby/TorqueBox throughput
* Running through the exercism.io Elixir tasks

I never really (and still don’t) feel like I know Elixir very well, but I am very excited by its promise and the robustness of the Erlang Virtual Machine.

Some of my current work at Puppet Labs involves creating automation to pull data from SalesForce, combine it with additional data sources, crunch the data, and report out. I am competent (not exceptional, but competent) with the SalesForce API and the databasedotcom Ruby gem.

I have an upcoming service that will need to access SalesForce data, and I have been writing it twice: once in Ruby (which will likely be deployed via JRuby) and once in Elixir. This may seem silly or wasteful, but I have certainly been far more deliberate about the service’s robustness as a result. Asking questions like
* How long are you willing to wait for a response?
* Can I tell you to go away and try again later?
* How is the load modeled, in terms of quantity, frequency, content?

Earlier this week I got to the point where I could no longer fake the SalesForce data, and I had to connect it for real. No problem for Ruby, just add gem 'databasedotcom'. For Elixir, hex.pm: nothing, Google: nothing. Let’s check Erlang. Well, there’s ErlForce, but it uses the old SOAP API *and* hardcodes a version that doesn’t even answer anymore.

Well, crap.

Should I write an Elixir client library? I don’t particularly *want* to write one. Screw it, I’m gonna write one…

A few hours later, I had a workable version where I could log in, figure out which SalesForce host I should utilize, see what data exists, introspect each of the SalesForce objects and their attributes (useful for dynamically creating database tables with the proper data types), and perform generic queries. In other words, 90%+ of the functionality I use in the databasedotcom gem. In a few hours! That’s mostly Elixir, very little Jeff.

I uploaded it to hex.pm, the Elixir/Erlang package repository.

No one else is using it, yet. Although, on the commute the following morning, I had a stunning and terrifying thought: Because no one else is accessing SalesForce from Elixir (as gauged by lack of libraries and questions about it), I am the best (and maybe only) in the world. This position would be trivial for someone like Fred Hébert to usurp, but, for now, I’m going enjoy it, and work dilligently to maintain.

How to: Trello 3-Legged OAuth with Sinatra

The ruby-trello gem does a decent job of illustrating how to thread-safe (as much as one can w/ Ruby) authentication, but practically nil on thread-safe 3-Legged OAuth. It took me far longer than I care to admit to get it actually working, but now that I have I thought I’d post an example application.

I primarily deploy my simple webapps on Heroku, so you’ll definitely see some Heroku-isms (e.g. Procfile) in the code repository.

You can see the code in action here.

How to get a job at Puppet Labs

I’ve been working at Puppet Labs for 4 months. It has been awesome! But, I almost didn’t get a job here. I’m going to tell you the secret to getting a job (or at least in development) at Puppet Labs or any open source company.

The interview process goes like this (and here’s the prep guide):

  • Initial interview with senior developer
  • Interview with VP of Engineering
  • Half day of 1:1 technical interviews
    • 2 technical interviews
    • Cross-functional interview (maybe with UX or QA or Ops, etc)
    • Pair programming
  • Interview with Luke

Yes, the CEO still interviews everyone we hire (and many we don’t). I think it’s awesome that Luke makes times for candidates. It makes him more approachable.

Here’s what it takes to get a job offer:

  1. Be awesome at something
  2. Make us passionate about you

Both are well within your power.

We used to have a panel interview in place of some of the 1:1 technical interviews. I had a ho-hum panel interview. I wasn’t mentally prepared for it. I thought I was coming in for a couple pair-programming sessions, not a panel and then pairing. So I was caught a bit off-guard for the panel, and it showed. It wasn’t terrible, just not enough to make them passionate about me. Fortunately, the NEXT day was a Triage-a-thon. I showed up and managed to close a couple tickets. I also continued to submit pull requests for open tickets.

I didn’t do it to suck up to the interviewers. I did it because I was trying to learn more about the product and the codebase. I had barely used Ruby or RSpec at that point and I had *never* submitted a pull request. I needed to learn them, and I was upfront about being a noob. This is what got me the job, not the interviews. Puppet Labs knew I was capable of doing the job, and the hiring risk was low.

That’s the key to making us (or any employer, really) passionate about you:

Show that you can do the job by doing the job.

We have a couple thousand open tickets. Find something that you think you can do and submit a pull request. Rinse, repeat, apply.

I look forward to the pull requests…

EDIT: updated with the refined interview process and official interview prep guide. Yea, feedback in action!

Solving Hanging with Friends (Pt. 1)

I like to play Hanging with Friends, and those that I play with are very competitive. I’ve been staying on top, but just barely. I’ve spent a lot of time informally deducing. I wondered if I could write a program to solve the games for me.

Where to start? It seemed like a big task…

There are two distinct activities to solving a hangman game:

  1. Finding a list of possible words
  2. Figuring out what letter to guess next

Each can be developed independently and then brought together for an effective solution.

There are many, many ways to do this, but I thought I’d just start hacking away in Ruby and then transition to something else at the pain points. Ater a couple hours of late-night coding, I found the Ruby-only solution to be good enough for personal use.

Part 2 deals with figuring out which letter to guess next
Part 3 focuses on how to find the list of possible words
Part 4 brings them together
Part 5 talks about next steps

Solving Hanging with Friends (Pt. 2)

Continued from Part 1, I’m off to write something to solve my Hanging with Friends games for me.

Because the two pieces of the puzzle (word list and next letter) are independent I can begin with either. I started with the letter frequency analysis because it seemed like a more tractable problem.

Ok, so it seemed easier but where should I start? Since I’m not totally clear on what it should do yet, I’ll start with the tests (aka how it should behave).

Given a list of viable words,

  • it should count how many words a letter appears in
  • it should only count a letter once per word
  • it should count each letter of the word

Here’s the full RSpec for those tests:


    describe "letter frequencies" do
      before(:each) do
        @words = ["lilly", "silly"]
      end

      it "should not return nil" do
        Frequency.letter_frequencies(@words).should_not be_nil
      end

      it "should return a hash" do
        Frequency.letter_frequencies(@words).is_a?(Hash).should be_true
      end

      it "should count the number of words with that letter" do
        freqs = Frequency.letter_frequencies(@words)
        freqs[:s].should == 1
        freqs[:y].should == 2
      end

      it "should not return a frequency larger than the number of words in the list" do
        freqs = Frequency.letter_frequencies(@words)
        freqs.values.count {|c| c> @words.length}.should == 0
      end
    end
    
    describe "letters per word" do
      before(:each) do
        word = "lilly"
        @letters = Frequency.letters_per_word(word)
      end
      it "should not return nil" do
        @letters.should_not be_nil
      end
      it "should return an array" do
        @letters.is_a?(Array).should be_true
      end
      it "should convert the letter to symbols" do
        @letters.count { |e| e.class == Symbol}.should == @letters.length
      end
      it "should only include repeated letters once" do
        @letters.count { |e| e == :l}.should == 1
      end
    end

Hmm, that’s a little more straightforward than I thought. Let’s make those tests green.

My first pass:


  class Frequency
    def self.letter_frequencies(words)
      freq = {}
      words.each do |word|
        letters_per_word(word).each do |letter|
          freq[letter] ||= 0
          freq[letter] += 1
        end
      end
      freq
    end
    
    def self.letters_per_word(word)
      chars = []
      word.each_char do |char| 
        char = char.to_sym
        chars << char unless chars.include? char 
      end
      chars
    end
  end

Ok, so more code to the tests than to the actual implementation. That seems about right...

Next we'll deal with how to make the word list contain only the words we actually want.

The most recent version of the code is available at GitHub: https://github.com/jeffweiss/hanging_solver.

Windows Service and Remote Desktop Mutex Sharing

Yesterday I ran into some weirdness while trying to implement System.Threading.Mutex synchronization between one of my Windows services and a colleague’s application.  The synchronization worked great in the debugger, but when running on a test system, the apps stomped all over each other.

Both had code somewhat like this:

  System.Threading.Mutex mutex = new System.Threading.Mutex(false, "SHARED_MUTEX");
  //some prep work

  //get the mutex for the shared resource
  mutex.WaitOne();

  //work with the shared resource

  //done, release the mutex
  mutex.ReleaseMutex();

Wha?!

I double checked the system mutex name, that was fine.  Didn’t seem like it should be a permissions issue.  The note in the documentation about Terminal Services kept nagging at me.  We weren’t running Terminal Services on the virtual test machine, but we were using Remote Desktop (Terminal Services lite, if you will).

I started by prefacing the mutex from my colleague’s app with “Global\”.  All of a sudden, the apps were working fine on the test system.  But I hadn’t changed my windows service app’s mutex name to look at “Global\”!

  System.Threading.Mutex mutex = new System.Threading.Mutex(false, "Global\\SHARED_MUTEX");
  //some prep work

  //get the mutex for the shared resource
  mutex.WaitOne();

  //work with the shared resource

  //done, release the mutex
  mutex.ReleaseMutex();

It turns out that because my portion was a windows service, it’s execution context changed from being part of the Terminal Services/Remote Desktop session, to being global.  The other app was still in the Terminal Services/Remote Desktop session context because it was an interactive application.  In order for it to lock on the same mutex, it had to explicitly look for the mutex in the Global namespace rather than the Local session namespace.

Makes sense, I guess, but still a little weird. Glad I didn’t spend all day tracking it down either…

How To: WCF Serialization Using Both XmlSerializerFormat and DataContractSerializer

On one of my projects, we’ve recently decided to switch from a SOAP-based interface for our SDK to a REST-based interface.  One potential problem: we can’t break the old SOAP interface.

Ideally, we’d be able to use the same underlying business objects for both access paths, but we were unsure how well WCF’s DataContractSerializer and XMLSerializerFormat would play together.  Our existing SOAP interface made heavy usage of the DataContractSerializer; I know we could have used that for the REST interfaces as well, but we needed to support XML attributes to align and integrate with other internal projects.

It turns out that you can mock up the business objects with attributes for both DataContractSerializer and XmlSerializerFormat, but the interfaces that expose these objects must be different.  So you can’t have, say, an IDataService that functions for both a DataContract interface and an Xml interface.  The attributes annotating the interface methods must be distinct, otherwise the DataContractSerializer will usurp the XmlSerializerFormat.


[ServiceContract]
[DataContractFormat]
public interface IDataService
{
  [OperationContract]
  List<MyObject> GetMyObjects();
}

[ServiceContract]
[XmlSerializerFormat]
public interface IDataService2
{
  [WebGet(UriTemplate="")]
  List<MyObject> GetMyObjects();
}

[DataContract]
[XmlRoot("myObject")]
public class MyObject
{
  [DataElement]
  [XmlAttribute("name")]
  public String Name { get; set; }
}

This may be not be ideal because now we have two interfaces. So, an alternative is to change to a single interface and pushing the attribute annotations to the implementations. That looks more like this:


public interface IDataService
{
  List<MyObject> GetMyObjects();
}

[ServiceContract]
[XmlSerializerFormat]
public class RestDataService : IDataService
{
  [WebGet(UriTemplate("")]
  public List<MyObject> GetMyObjects() 
  {
    return new List<MyObject>();
  }
}
 
[ServiceContract]
[DataContractFormat]
public class SoapDataService : IDataService
{
  [OperationContract]
  public List<MyObject> GetMyObjects()
  {
    return new List<MyObject>();
  }
}