Prophecy: Apple of Data Engineering with Raj Bains

Prophecy is a complete Low-Code Data Engineering Platform for the Enterprise. Prophecy enables all your teams on Apache Spark with a unique low-code designer. While you visually build your Dataflows – Prophecy generates high-quality Spark code on Git. Then, you can schedule Spark workflows with Prophecy’s low-code Airflow. Not only that, Prophecy provides end-to-end visibility into your dataflows with Metadata Search and Column Level Lineage. 

For Enterprises, in addition to developing new workflows, data teams also need to migrate thousands of old proprietary ETL workflows to the cloud. For that, Prophecy has built a Transpiler that automatically converts AbInitio, Informatica, SSIS and Alteryx workflows to high-quality Spark code. Learn more at

In this episode, we speak with Raj Bains, who is the founder & CEO of Prophecy.

Previously, Raj was the product manager of Apache Hive at Hortonworks through the IPO. He also headed product management and marketing for a NewSQL database startup. 

Raj continues to actively code in compiler and database technologies. His engineering roles include developing a NewSQL database, building CUDA at NVIDIA as a founding engineer, and working as a compiler engineer for Microsoft Visual Studio.

Full disclosure: Prophecy is a sponsor of Software Engineering Daily.

Sponsorship inquiries:

Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to to get 15% off the first three months of audio editing and transcription services with code: SED. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.