If you look at the internal architecture of the technology powering Uber, Pinterest, or Airbnb, you’ll see Kafka quietly plugging away. The technology spun out from when two LinkedIn engineers built the streaming platform to keep pace with the vast amount of data on the social network.
Jay Kreps was one of the two former LinkedIn engineers, and he’s now at the helm of fast-growth tech firm, Confluent.
The company provides a streaming platform separate from, but based on Apache Kafka, and its growth in its home base of Silicon Valley has led to it expanding into Europe via London.
“It started as a technical project that helped LinkedIn harness the vast data it had. By the time we left LinkedIn there was a trillion messages going through Kafka. We open-sourced it and it became well known technology in Silicon Valley,” Kreps tells GrowthBusiness.
We saw potential to be in every company, not just Silicon Valley tech giants. We decided to build around Kafka, but specifically for enterprise. We’ve expanded to the UK last year, and have seen explosive growth since.”
Most of Confluent’s growth can be attributed to good old fashioned word-of-mouth marketing.
“Unlike most other tech companies, we really came up with a very different way of thinking about data and because the technology is open sourced, we’ve seen teams adopting it in some of the biggest companies in the world. It’d be very hard if we had to knock on doors and tell people about how awesome this new technology is. That’s not how engineers work,” he explains.
“This is something engineers in companies are already excited about. With developer-centric technology, people talk to each other and discover new technology through their interactions with other engineers. It’s the best way of getting engineer buy-in, rather than showing up with a sales team to teach people.”
Confluent’s exponential growth was fuelled by three funding rounds, with an initial $6.9 million round led by Benchmark, a $20 million series B round led by Index Ventures, and it’s most recent being a $50 million round led by Sequoia Capital.
Existing investors Benchmark and Index Ventures also participated in the round and Sequoia partner Matt Miller joined the Confluent board. With total financing at $80 million, Confluent will use the new funds to further its vision of placing its streaming platform at the heart of every modern business.
“It’s exciting, if you step back from how it’s used,” Kreps adds. “All these companies are becoming fully digital. Traditionally, software was just something retailers or media companies bought to use on the side. Now these companies all have digital systems that run on data, and the question is now how do they use all of this data.”
For Kreps, Confluent’s technology can work across any sector. Citing the example of connected cars, he explains that all of the main car companies are looking into the technology, but there’s a whole feedback loop from the usage of a product that needs to be understood. connected cars. “This is a really big change from what it means to make a car. Some of the biggest car companies in the world are building off Kafka.”
In financial services, most organisations are built on real time data. Now these organisations can operate at scale instead of off a single computer, which is huge, says Kreps.
Similarly, big retailers are dealing with large amounts of data, and they’re competing not just with each other, but also against companies like Amazon who can work much much faster.
“It plays itself out very differently in each industry. We’ve seen use cases in shipping, IOT, banking, retail, media–really, it’s across the board,” he adds.
For a company that is growing as aggressively as Confluent, attracting and retaining talent is a matter of survival. “Some of our earliest employees were in the UK, and they’re engineers. The technology is so exciting for engineers, so from pretty early on, we’ve attracted the right talent,” he says.
“We’re working as fast as we can to build up a sales team, and bring out technical people. We have a pretty good set in London today, but we need to expand to support new customers. That of course means bringing in some of the best technical people.”
The Confluent team is 100 employees strong now, and according to Kreps, each person on the team fits his hiring criteria. “We look for people who are smart, because we’re solving a hard problem, but also humble. You can be smart but a little hard to work with, and that’s not good for the team. We also look for people who are empathetic, who can communicate with each other and can form a strong team. The type of system we’re building is a team effort. We look for people who are builders who want to come into a new company and help build it out.”
There are some people who are really excited by the opportunity of working with a growth company, says Kreps, and he often recognises that passion and drive in the first few minutes of taking to somebody.
“It’s very competitive, both in Silicon Valley and in London. Talented people have a lot of options, so I think the company we have is amazing, and the technology is the hardest and most interesting. The funny thing is engineers like challenges, so they’re most attracted to something really new and different, that has no immediate precursor that can be copied. Our offering helps drive the right type of person to us,” he says.
“Ultimately, our technology could be as big as data warehouse is as a category. But for that, we need to pace ourselves and attract the best talent.”
