Ruby, ActiveMessaging and ActiveMQ

Just wanted to have a quick post on how we switched to ActiveMQ for our ruby app. We ran into problems that were easy to fix though hard to see where the error was. The first was a problem with idle connections and firewalls, and the second was how slow consumers ‘ate’ off the ActiveMQ queue.

In our situation, the pollers/consumers of the messages exist in the DMZ, and the ActiveMQ server is behind an internal firewall. This firewall drops connections that are idle for >1 hr. ActiveMessaging, for some reason I still do not understand, does not re-connect in this situation, though ActiveMQ sees the consumers as not connected. And you can’t have ActiveMessaging clients ‘ping’ the server… no such API request possible. Suggestion on the web to fix this: Create a ‘keep-alive’ topic that you ping once in a while. Since its a topic and not a queue, all listeners get the ping. No idle connections now. (In our firewall, you cannot turn of it behavior on a single port, so we opted to go for the keep-alive topic.) Is this optimal? Not really, but it does work.

The second issue was ‘greedy’ consumers. If we pushed 10 messages into a queue, and the queue had 3 consumers, one consumer would get all ten. If the client died after the first message, the other nine would be lost. Solution was to add the “:ack=>’client’” header, so the client acknowledges only after its done processing the current message. Second issue, if you started up one consumer and 10 messages were in the queue, then started up two more consumers, the first client again would only process those 10 messages. Though if it died after the first message, then ActiveMQ would give the other messages (redeliver) to the other consumers. Solution was to set the activemq.prefetchSize to one. Both of these are done on the processor’s definition, not in the configuration file for ActiveMQ. The magical incantation is this:


subscribes_to :mail_event, :ack=>'client', 'activemq.prefetchSize'.to_sym=>1

The hard part was that in ActiveMQ, you have several different prefetchSize options, but the correct location was in the stomp section as its protocal dependent, and then ‘where’ to set it in ActiveMessaging got confusing. But it now works like a champ.

Goldman Sachs has record profit this year?

If the news is to be believed, Goldman Sachs had a record profit this ‘year’ and will be paying out the largest bonuses ever. To quote the original article:

“…it seemed inconceivable that a firm owing the US government $10bn would be looking to break all-time records in 2009.”

I’m completely disgusted especially since I heard this on NPR right before they start talking about how businesses in general are reducing retirement benefits for their workers. Ironically, the reason for Goldman Sachs’ huge windfall is the poor economy. The government has to use financial institutions to move bailout money to companies, even though the same places help cause the original problem in the first place. And of course, now the Goldman Sachs paid their ‘TARP’ money back, they can set their bonus level and executive pay to whatever they want.

Something is fundementally wrong with the business model in the US. The power of these financial institutions are only getting bigger as the whole economy weakens.

Palm Pre ‘hard close shutoff’ fix…

One hardware issue seen with the new pre is that if you close the keyboard ‘firmly’ it seems to jiggle the battery enough that the disruption in power causes the pre to shutdown completely. One person’s fix was to put a piece of paper between the pre and the battery so its tighter. Another was to put folded electrical tape there.

Here’s a third idea. There is a tab in the pre when you open the back that you pull to get the battery out. If you fold that in half and replace the battery, you can still use the tab to remove the battery, and the folded tab is enough to ‘lodge’ the battery tighter in place so the shut-off doesn’t happen.

I have this idea…

I have this idea on what application I want to write for the Palm Pre. It’d be fine for the other smartphones too, but I have a pre, so that’s what I want to write for. But the damn Mojo (Palm Pre Developers Kit) is still unavailable for public consumption…

Just had to complain… open up Palm… we’re waiting…

Palm Pre calendar

For the last weekend, I couldn’t see my calendar on my pre, even though it was alerting me to my events. Finally figured out that the problem was I had too many calendars. My work Google calendar lists 20 calendars in total, and home has about 8… so the display was just over-whelmed. On the pre I just told it not to display most of the calendars in the ‘all calendar view’ and everything works fine now. Of course, then I removed calendars from my other accounts that I don’t care about anymore either.

Palm Pre… day one

So I stood in line for a palm pre… two actually. My current plan has the 1500 family with unlimited data plan already, and the phones my wife and I use contract was ready for new phones, so we were good to go for new phones.

Decided to go to Sprint store rather then risk BestBuy not having enough. Good choice as it turns out. 150 in stock at Sprint, and like 2 at BestBuy. With over 200 people that I know about showing up at the small store I went to, it must have been good for Sprint to see that volume move. (Though now I have to deal with the mail-in rebate, which is lame)

Palm pre itself is fairly easy to use. Comments made about the sharp edge when opening the pre is no joke, but easy to deal with. The OS is interesting and while learning the ins-and-outs of it, I think I now get the ‘palm’ and ’sprint’ business plan with them trying to recover.

First palm… this is a good phone, and I can see why some say its an attempt at going after the iPhone. But really what palm is doing is what Google should have done with their G-Phone… it integrates with websites easily. All my mail accounts (yahoo, google, corporate which is basically google for me) are available. All my contacts (not yahoo for some reason, but that’s fine with me) are there, faceboook integrates easily, etc. WiFi makes it fast at home and (likely) at work on Monday. Really, this is a better google-phone. They didn’t really try to one-up the iPhone by ofering what it has and more. (iPhone is just an extension of the Apple hardware suite. Google-phone is an extension of the Google/web suite) Which makes me think that Palm wants to be bought by Google. If anything, this phone proves it. (Yeah, its not android, but there is no reason the ’software/website’ integration that webOS makes easy couldn’t be done in Android. Though I can’t wait to see the Mojo API and start making my own webOS apps.)

Now Sprint. The service they give now is better, and starting with the palm pre gives people less reason to go to AT&T or T-Mobile… so this reduces the stream of people leaving. But what I noticed is that the plans are much cheaper on Sprint compared to AT&T and T-Mobile. I had a 20% discount on Sprint from working at Thomson. When I left, I was able to keep that discount, so I never left Sprint. Now with a CostCo membership, you can get a 23% discount. Anyone can get that. What the palm pre release on Sprint does is get Sprint’s name out there again… which will also make their discounts more known. Palm will be releasing on other providers by 2010, so Sprint’s discounts will make them more palatable now.

After a few more days, I’ll post about using the phone itself. Right now, webkit works like it does everywhere else. The fact that speed-dial is replaced by universal search will take some getting used to. (Or figuring out how to set speed-dial) Tweed (3rd party twitter app) is ‘okay’ on the phone, and the New York Times app is a good start. Of course, the task list I’ve already started using… and I await the day that it syncs to Google tasks. (Oh man, that would be awesome!)

More later…

Migo Experiment…

I’m starting my evolutionary programming again after a 2+ year hiatus. I decided to change my approach somewhat. Some of you may remember that my EP swarm I built was referred to as ‘my bees’. However, they were somewhat suicidal for reasons that I still do not understand, and the results they gave were useless.

My new approach will still use the ’swarming’ technique, but this time the goal of the system will be to optimize an evolutionary computing neural network. This makes the ‘hive mindset’ I was programming before a bit more structured and less chaotic. Because of this structure, and the system is now truly a data-mining application, I decided to call my new implementation of the ’swarming’ technique ‘Migo’ after the Lovecraftian Migo.

In this system, each computational node can evolve and the neural pathways can be taught. In the bees approach, each program evolved and acted alone. The ‘hive’ was constantly changing in fairly random ways and never converged correctly. In the Migo approach, the neural network is the end goal and the number of components that can evolved are smaller, which should be easier to see if its working and what changes to make.

I’ve always enjoyed my bees and I will try to get them working again one day. Though I’m hopefully the Migo experiment will have better results.

Rcov versions…

Rcov is a great tool to verify what code is touched by test cases. Its only C0 testing, in that it checks that the code is hit for a test but not testing for all possible permitations.

The original project is all but abandonded and several forks on github.com have come out. The latest one that I like the most is the relevance on which is based on Spicycode’s fork.

The Dell Mini9

So the Mini9 came today, and I moved it into place on my wifes desk. Its a pretty nice machine. This model has the faster processor, 2 Gigs of ram, 32Gig hard drive and blue-tooth support. At some point well get her a blue-tooth mouse and keyboard, but for now we’re using the one she used to have. Also, we hooked up her old monitor as the external monitor.

As sad as it is to say, its pretty much as powerful as her old system, with a few differences. Its a touch slower, but runs much of the same programs she’s use to just fine (Firefox, open-office, pidgin). The keyboard is fantastic, though a little small for my hands. Its the perfect size for Jacob, who much to his disapointment, as to settle for Kirsten’s old computer.

I’ve include a second picture so you can see how tiny it is compared to a 17inch monitor. The box it arrived in was smaller then a book sent by Amazon these days. Honestly, I want to get one for me now.

Sun may live…

It looks like Big Blue may purchase the Sun. This would only be good for Sun and specifically, benefit Java developers big time, as IBM is really one reason why Java is as successful as it is today.

But at my core, I wonder what would happen to the recently purchased MySQL folks compared to DB2 and the PostgreSQL support team. I have no clue on how IBM’s very strong database group would view those segments.