Four-year-old startup Databricks raised another $140 million to solve the hardest problem in AI

He had initiated discussions with investors in the same week he did back-t0-back keynote speeches at two big tech conferences, in two different cities, one of them sponsored by his own company. During that week, investors "were just bombarding me with phone calls," he describes. Less than four weeks later, he had a handful of term sheets, and by eight weeks, "the money was in the bank."

VCs were crawling over themselves to grab a bite of Databricks for a one main reason: In just four years, Databricks had already amassed about 500 big companies as customers, so revenue was growing, although

The other reason is that Databricks founders have also become famous in their field.

They invented a big-data technology called Spark that's become extremely popular with enterprises because it helps computers chomp through vast amounts of data super fast. This, in turn, makes it easier to build machine-learning and artificial-intelligence apps, which require computers to chomp through large amounts of data very fast in order to make decisions.

Crossing the Valley of Death

The funny thing is, the six Databricks founders never intended to be entrepreneurs. They founded this company pretty much out of frustration when the established enterprise software industry dissed their work.

"We were researchers at UC Berkeley and we really just wanted to get this tech out. We were hoping the world would just pick it up. We talked to all the existing companies out there and told them, 'we developed Spark, we just want to make these machine learning predictions and big data software as simple as possible.' And actually none of them picked it up," he recalls.

The companies they talked to all poo-pooed the tech as "academic software coming out of a university," he remembers, and said "we don't think Spark will be anything enterprises will use."

They were facing what famous researcher and Google X founder Yoky Matsuoka calls the 'The Valley of Death" in technology development. That's where tech developed in a research setting can't make the leap to a commercial product because the researchers don't have the experience or interest to do it (they want to find the next breakthrough, not iterate on a past one). Meanwhile, commercial vendors don't have the time, experience or interest to take raw tech and figure out how to turn it a product (they need to focus on their sure-fire revenue generating products).

So, in 2013, these researchers just up and launched their own company. Andreessen Horowitz's Ben Horowitz saw the promise, invested $14 million, joined its board and advised them on how to turn it into a success.

The founders still gave Spark away as open-source software, meaning anyone can use it for free and contribute to it. And Databricks offers a fully supported commercial version.

Solving the next big challenge

With the new influx of $140 million, Ghodsi and team are hoping to tackle the next big problem in the big data/machine learning/AI world: the lack of trained people.

They have just launched a new product that Ghodsi describes as "Slack for AI."

It allows teams to work on a project together, including the business folks, the AI scientists, and the programmers. The actual AI app can be developed by all of them from inside this collaboration tool, called the Unified Analytics Platform.

Ghodsi describes his company like one big, happy four-year-old rocket ship.

"It's been a blast these past four years, where we went from zero to this and all six cofounders are still with us. Most cofounders get into fights and backstabbing and 2 or 3 leave and they hate the company and they erase their names from the web page," he laughs.

But because Databricks found an untapped niche in big data and AI, it quickly generated revenue, which led to investment, which led to growth, which led to happy cofounders.

In addition to the six cofounders, (Ghodsi, Matei Zaharia, Ion Stoica, Patrick Wendell and Reynold Xin), Databricks employs 220 people, Ghodsi said.

The new round was led by Andreessen Horowitz, with participation by New Enterprise Associates and Battery Ventures.