BDWAI London Conference 2022

Subpage Hero

Subpage Hero



Use case: From batch to real time – how Compare the Market is building a next generation data engineering platform

02 Mar 2022
Data Management & Integration

The switch from batch processing to real time processing can be a daunting one but with the right approach and the right architecture, it can be highly rewarding for both the customer, the business and the engineers building it. Attend this session to learn how CtM has:

• Built a scalable, microservices driven platform for delivering personalised real time content to our customers
• Reduced downtime and incidents by moving away from overnight batch processing
• Used Spark to process our historical data loads
• Constructed our next generation data lake

  • -
Torquill Pagdin, Senior Data Engineering Manager (Marketing Data Engineering) - Compare the Market

2022 Sponsors

Platinum Sponsor

  • Neo4j
  • Mongo DB
  • Seldon


Gold Sponsor

  • Softserve
  • Domo
  • WhereScape
  • Treasure Data
  • H2O
  • Snowplow Analytics


Silver Sponsor

  • Altair
  • Influx Data
  • Trifacta


Theatre Sponsor

  • FiveTran
  • Pyramid Analytics
  • Starburst


2022 Partners

Media Partner

  • AIgents
  • Computer Weekly
  • Cloud Industry Forum
  • Computer Weekly


Media Partner

  • WeAreTechWomen
  • DataFloq


Media Partner

  • IOT Now
  • Digital Agency Network
  • DAN Institute
  • Trusted Computing Group


Media Partner

  • The EE
  • CIOInsights
  • Packt


Media Partner

  • CIO Applications Europe
  • Opinno


Media Partner

  • CIOReview
  • Sustainability Magazine


Media Partner

  • Top Business Tech
  • TechUK
  • EM360


Newsletter Sign Up