Track Chair Interview: Steve Loughran

Continuing our series of interviews and viewpoints on Hadoop Summit Europe. This interview is with Track Chair for Committers – Steve Loughran, Member of Technical Staff, Hortonworks. You can register for Hadoop Summit here, and see the detailed schedule here.

HS: Tell us a little about you. What parts of Hadoop are you currently working on?

SL: In my 2 years here I’ve worked on high availability Hadoop, cloud storage and openstack integration, and the YARN service architecture. I’m currently involved in a project to host long lived applications in a YARN cluster. If these things sound exciting, come and join us!

HS: Tell us a little about your track. What is exciting about it this year?

SL: It’s the committer track , so it’s about things being worked on by fellow committers – colleagues, friends and more. It’ll be technical -we made sure of that in our choice of talks- and go all the way up the stack.

HS: You saw many submissions. What themes emerged amongst the talks in the track?

SL: A key theme was so much is going on. HDFS is evolving -support for storage layers such as SSD and in memory caching is going to boost performance of low-latency applications

YARN now lets people write new applications & explore new algorithms in a Hadoop cluster – it’s worth attending those talks to see what new execution concepts are developing beyond Map and Reduce. Hbase-wise there’s exciting work going on – that’s going to be covered, then there are the layers above. There’s a lot of stuff, enough that its hard to keep track of. I hope to bring myself up to date with a lot of this work myself.

HS: Who were the judges in the track?

SL: Jakob Homan, Hadoop PMC (Project Management Committer) member  at LinkedIn; Lars George, HBase committer who works at Cloudera and lives near Stuttgart, and Isabel Drost – Mahout committer and Berlin Resident.

HS: What’s your perspective on Hadoop in 2014? What are you hoping to see?

SL: There’s more contributions coming from Europe itself, it’s a measure of adoption here, and means it’ll be great place to meet up with developers that I’ve got to know round the continent.

Hadoop-wise, I think we’re seeing things moving beyond data-analysis operations to applications built to run inside Hadoop clusters – it’s a driver for low-latency work, be it SQL queries, streaming, HDFS storage layers, and others. Now that YARN simplifies running applications within a Hadoop cluster – it’s becoming easier to write applications that build on it. Alongside that more people have experience with the technologies and more frameworks are appearing – such as Spring for Apache Hadoop. I’m hoping to see new ways to use the platform.

HS: What sessions from other tracks are you most interested in seeing?

SL: I’m looking forward to a lot of the operations talks. Allen Witteanuer’s “Operations at LinkedIn” was a classic last year. He’ll be back, along with many others. Again, this shows a maturation of the technology – and the fact that organisations like eBay, Spotify and Xing are willing to share their experiences means that it’ll be interesting.

Mainly, I’m hoping to see how people round Europe and nearby are using Hadoop.

HS: Thank you for all the efforts you and judges gave to making Hadoop Summit Europe 2014 a great success. We’ll see you there.