This post describes an old Deutsche Telekom project around high-volume Smart Home data. The work was a good example of starting with a narrow proof path: make the core data flow work, measure it, and then use that evidence for the next decision.
Telekom, as they prefer to call themselves, needed a system that could seamlessly handle a staggering one billion transactions per day, all while tackling the complexities of data anonymization. But here's the twist: they still wanted to extract valuable insights through statistical analysis from this anonymized data. Challenge accepted!
We got the request in late December from an innovation department at Telekom. They needed to convince upper management that it was possible to build a system that could handle this amount of traffic within a reasonable budget. Problem was, that this proof was needed by January to be approved in the next year's budget.
In three weeks we built a proof of concept with Erlang and Kafka that could process the target traffic shape and anonymize the data before analysis. That gave Deutsche Telekom enough evidence to continue the project and budget discussion with a concrete system rather than a slide deck.
Unfortunately there was a political policy decision at Telekom at the time that only allowed the use of three programming languages in production. Erlang was strangely not one of them, but Java was. So over the next three months, we worked tirelessly to transform our Erlang proof-of-concept into a production-ready solution in Java. It still used the same Kafka setup, anonymization and Grafana monitoring but some of the end points were replaced by Java, and the corporately approved Spring Boot framework. This required a bit more work but HappiHacking's team of experts designed an architecture that could handle the massive data volume, ensuring smooth transactional systems capable of handling billions of messages each day.
This innovative data streaming solution paved the way for Deutsche Telekom's own team to build new services upon our foundation. Among those services was the MagentaZuhause App which received a reddot award.
As a little side project we also managed to get data from a Magenta TV set top box which was a black box to us. By combining bash scripts, C, regexps and some innovative hacking we manged to get information from the TV, and we could also turn the TV on and off remotely within a few milliseconds. This proved that we could build a service for remotely managing a TV in the home. For a real product we would of course build it into the software which another department within Telekom had access to, but such a cross departments project could take months to get started.
" - Why did the smart TV get turned off?"
" - Happi's joke was so bad."
Throughout the project, our partnership with Deutsche Telekom flourished. Together, we navigated the intricacies of the Smart Home landscape, crafting solutions that blended robust data handling with GDPR compliance. Our motivated HappiHacking team went the extra mile, delivering qualitative results while sharing a few bad jokes along the way.
In the words of Filiz Hazer-Yilmaz, Deutsche Telekom's representative: "If you are searching for a motivated team that will go the last mile with you, deliver qualitative stuff, and understand bad jokes…HappiHacking is the right choice!"
At HappiHacking, we thrive on challenging projects that push the boundaries of innovation. Our Smart Home collaboration with Deutsche Telekom exemplifies the power of rapid prototyping, scalable systems, and a touch of Swedish irreverence. We're proud to have played a part in designing and implementing new IoT services, revolutionizing the Smart Home experience for Deutsche Telekom's customers.
