Secrets of BackType's Data Engineers

We've been very busy lately at BackType. We moved into a new office, welcomed two new members to our team and have been furiously iterating on our products and technology (occasionally stopping to enjoy some beers and emerging programming languages). On the tech side, we're working hard on a new stream processing framework to help our customers make sense of social media in real-time. Right now we process more than 100 million messages a day, both in batch and real-time, but our stream processing is somewhat hand-coded. Nathan has been leading the charge on our new framework (code-named Thunderlog) that will generalize the real-time work-flow much like how Hadoop does for bactch work-flows.

A few weeks ago we sat down with ReadWriteWeb's Pete Warden to talk big data, stream processing and BackType's data pipeline. We had a blast geeking out and now he has published a very flattering article about our technology on RWW. Thanks Pete! Check out the article; if what we're up to sounds cool get in touch, we're looking for badass product and data hackers (http://www.backtype.com/jobs).

Permalink | Leave a comment  »

Source: http://tech.backtype.com/secrets-of-backtypes-data-engineers

computer repair tools computer repair training computer repair courses computer repair jobs