Episode 42: AI, Tech Debt, and the Future of Engineering

Welcome to SphereCast, the podcast for forward-thinking tech leaders. In today's episode, how AI is transforming data-heavy applications and helping tackle tech debt head-on.

Welcome to this episode of SphereCast, where we go beyond the buzz to uncover what's really happening at the intersection of AI engineering and enterprise tech. Today's conversation dives into how AI is reshaping the way we build, scale, and maintain data-heavy applications, from reducing technical debt to helping engineers move faster and smarter.

I'm joined by Alex, a technology leader with deep experience in the telco space, to unpack what's real, what's next, and what's possible. So, hi, Alex. Welcome. I introduced you very shortly, very briefly, but please go and please add a little bit more to who Alex is.

Hi, Edin. Thank you for having me here, and thank you for the good and brief introduction. Just to shed a little bit more light about myself, I'm a technology leader with more than a couple of decades of experience. Grew up from a software developer to executive ranks. Been in technology consulting all my life. Worked with both large and small organizations, including large and small enterprise clients, and currently I'm a Field CTO with Sphere.

Well, great to have you here, Alex. And I would say without further ado, let's let's dive into some some spicy questions, Alex. So, many companies struggle with technical debt, but it's not just about messy code. It slows down the business. So, do you see AI playing a role in helping engineering teams identify, prioritize, or even resolve tech debt? Any examples you've seen this in real life in the wild, so to say?

The simple answer to the question is yes, AI definitely can help with resolving technical debt. But before we even talk about AI, you have to take a step back and really understand first, why do companies have technical debt? All that really boils down to is for organizations, as they continue to evolve through their business development journey, they continue to pile up on applications without really considering the future.

What that really means is people, instead of really developing robust architectures and robust platforms that will be able to sustain rapid change and evolutions, they usually do a lot of patchwork. Some of that happens just because things evolve too fast, some of it happens because people don't implement the correct processes within their software development lifecycle. But whatever the reason is, most organizations are struggling with technical debt.

In reality, every organization is trying to continuously minimize their technical debt. And now, with evolution of AI, a lot of the toolings that help you to modernize your platforms really embed AI capabilities that help you to, one, analyze your code bases for unnecessary technical debt, and further, they allow you to essentially go and implement these modernizations to reduce the technical debt. Whatever path you choose, however you are trying to approach that, using the right tools definitely is going to help you.

But I'll continue to tell people that tooling is not the only answer or it's not really the only solution. You definitely have to ensure that you are also implementing the right engineering processes and principles within your organizations to continuously reduce technical debt and continuously ensure that you are introducing less technical debt as you continue to evolve your platforms and technical ecosystems.

All right. Well, that that was a lot to unpack. But, so, basically what what I heard definitely is yes the answer, the short answer, but there are no shortcuts, as as as it seems, right, with with using AI.

That is that is correct. Unfortunately, in the world of technology, there is no such a thing as a shortcut. There are definitely tools that will help you to do things better, faster, but you still have to go back to the basics and make sure that your software engineering practices and processes and governance are in place to ensure that everything is being done in a coherent manner.

All right. Okay, thank you for that, Alex. Let's go maybe to the next question. So, for companies dealing with large volume of user-generated data or content, moderation becomes a challenge. So, how do you think AI can help with this, and where do you believe human oversight is still critical?

It's a hard question to answer, and in order to really understand that whole concept of big data, you have to essentially go a little bit back in history to look at how big data evolved. Once technologies like Hadoop, Spark came to life, a lot of companies realized that they were essentially able to collect and process large quantities of data. So, for example, companies like Visa that processes credit cards, they were essentially able to collect all credit card transactions, store them in specific data lakes, and really run analytics to figure out what are people buying, what ages are spending on, what genders are spending on what, so that way they can essentially use that information and really monetize that information by providing that type of data to marketing organizations.

For example, if they were seeing that younger generation is more susceptible to spending, for example, on new technology, such as the latest iPhone, while the older generation was more spending, for example, on specific clothing items, they would use that information and figure out how to monetize it. So, that's how big data came to life. But unfortunately, with all the hype around big data at that time, when people were creating their data lakes, they were doing it in a fairly unstructured way. And essentially, from actual data lakes, what we came to is essentially data swamps.

So, with all the data swamps around there, and each organization has more than one data swamp and probably stored in different ways, shapes, and forms, the question essentially becomes, well, how do you continue to make use of that obscene amount of information that each company stores? So, everybody's asking the question, well, how can AI help?

And in reality, AI can help work with large datasets as long as the datasets are coherent in such a way that AI can essentially make use of that. In order to do that, there are multiple ways. One is potentially create an aggregated data lake or a data abstraction layer where you're essentially pulling some information from various data lakes for AI to be able to more coherently process it. Or you essentially go through a data modernization journey where you essentially modernize your data swamps to be more really like data lakes, whether it's structured data or unstructured data, but you make your data more susceptible to AI use, because at the end of the day, even if you have your AI models, but it takes you 10 seconds to come up with a particular piece of information, but that's, let's say that information is being displayed on a website, all users don't like to wait for 10 seconds while the wheel is spinning on the website.

So, with that said, you have to undergo some sort of a data modernization journey. Now, where human is still needed is really to, one, figure out what the structure of the more coherent data lake is going to be. Two, ensuring that you have the right data governance, so that way, in the future, as you continue to pile in more data or create new additional data models, you continue to have a data lake instead of really that data lake becoming a data swamp.

You create data marts. In other words, you create data as a service, where, for example, if you're storing large datasets, you essentially figure out how you're going to give different datasets to different personas, whether it's internal or external, in a more coherent way, right? So, only the right people see the right data and actually get it in a fast and applicable fashion. And I know I can continue to ramble on and on on that, but that's the basics. And happy to dive deeper if needed.

All right. All right, Alex. No, it's very interesting. And, of course, you know, we hear it all the time that when when customers come to us that, you know, AI is the thing that they think of, but actually there is there is first stuff that needs to be done, and that that is definitely the data modernization and putting things in order so that AI can be used as well, right? So, so makes makes complete sense. Let's go let's go to the next one, Alex.

So, we often hear about growing too fast or systems breaking under a sudden scale. What's the hidden cost of not planning for scalability early, and can AI help spot bottlenecks before they become incidents?

That is a question that every operations manager is asking themselves, because everybody who is building a particular infrastructure, they're building it for a particular purpose. Let's just take an e-commerce application, and let's say that e-commerce application is hosted on the cloud. One of the reasons why people would choose to host an e-commerce application on the cloud is because cloud provides you infrastructure as a service and it enables to essentially scale your infrastructure based on need.

Now, that's all great in theory, but in practice, unless you build your e-commerce platform in a way that actually can take advantage of scale-on-demand infrastructure, that's essentially not going to help. So, you definitely have to spend enough time to architect your platforms and applications in such a way that they can actually scale together with your infrastructure.

Now, if you're not going to do that, you most likely going to run into problems. And one good example from the telco world is essentially the iPhone launch that happens in September. Apple releases its latest and greatest products in September, and an obscene amount of people are hammering on all the telco websites, whether it's verizon.com or att.com, to go and try to buy the latest Apple technology.

Now, if those websites are not built to a point where they can, specifically for the Apple launch, to handle that amount of people, and that's, we're talking about hundreds of thousands of people hammering on those websites, then these companies are essentially, one, going to lose their reputation, and two, they're going to lose hundreds and thousands of dollars, if not millions of dollars, in potential revenue. Because if, let's say, verizon.com is down, well, I'm going to go to att.com and I'm going to try to buy that iPhone. And when each iPhone costs more than $1,000, 100,000 people, do the math. That is obscene amount of money.

So, when we're talking about AI, there are a lot of what we call AIOps tools on the market that essentially allow you to both understand how your current platform is behaving. It allows you to model how a platform behaves under stress, and it also allows you to essentially make remediations on the fly. Some of the more common tools are, for example, Dynatrace or New Relic. They're definitely some of the big names within the AIOps space. And regardless whether you're running things on the cloud or on-prem, these tools help you to ensure that your platforms and applications are behaving as needed under stress and duress, and make specific adjustments, both either to resources that are needed for application or infrastructure, or even just alerting you that there are anomalies and you need to take care of that.

That's definitely insightful. We will put the tools also in the description of the podcast, so so people can can definitely check those tools out. So, to the next one, Alex, and it's about the AI as a co-pilot for engineering teams. So, there's a lot of talk about AI-enhanced dev tools, from code suggestions to auto-documentation. In your view, how much of this is hype versus actual help, and what's been your experience with these tools so far?

It's an interesting question. At the end of the day, before answering specifically about dev tools, let me give you an example. If I give you a Ferrari, it's a fast car. I'm pretty sure you'll be able to drive faster. But if you don't have your driver license, then it doesn't matter how fast of a car you have, you will not be able to drive. The same applies really to dev tools.

Dev tools is not something new. Dev tools have been around forever, even with the simple introduction of IDEs, such as such as Eclipse. The actual development cycle became faster, because people were able to see their code, get code suggestions, et cetera. Same thing with AI tools for development. You're able to generate code snippets, you're able to do development in a faster way because you might not need to essentially go and write mundane routines, such as binary sort or anything of that sort.

But as engineering resources or as software developers, we still have to essentially have our software development education. We still have to know the basics, because regardless of how much the AI tools are able to help you to do your job faster, unless you do spend time on education, making sure you have your computer science degree, making sure that you have experience in writing applications to the specific use cases that you're developing, whether it's front-end, back-end, full stack, specific languages, AI tools are not going to help. So, again, it's chicken and egg, because I keep hearing people saying, "Oh, now I don't need developers. I'm just going to have a AI tool write code for me."

Unfortunately, that's not true. Maybe sometime in the future we'll get to a point where somebody speaks English and a tool translates that English into machine code and everything is going to be nice and peachy, but we are very, very far away from that. So, the only thing that I'm going to continue saying is that we still need software developers that have proper education, proper experience, understand what they're doing, whether it's embedded development, UI development. They still need to understand what they're doing. And yes, AI tools will make their job easier by being able to write code faster, but the engineers still have to understand what code they're writing.

Thank you for that point of view. I think, if I may call it, the internet is split into into that point of view. So, yeah, very, very interesting. Yeah, I I saw also some examples of companies that are actually rehiring now people, as Klarna, you know, one of them, because they they were betting too much on it. So, as you said, I think I think, I'm sharing a little bit of your view that in a future we maybe a little bit closer to that to that vision, I think. But at the moment, it doesn't seem like it, yeah.

Alex, let's next question is also our last question, maybe. So, it's about future-proofing enterprise apps. So, I would like to know, as AI becomes more embedded in infrastructure and apps, what should tech leaders be doing today to prepare, and how can we integrate AI into our product roadmaps without creating more complexity or technical debt? I lied. It's two questions, actually.

Good question or good questions. I'll continue going back to what I call go back to the basics. So, in my opinion, in my professional opinion, in order for somebody to ensure that something is bulletproof for the future, in reality, there is no such a thing as bulletproof for the future, but you can still build your applications in such a way that they will be more susceptible to change, whether it is more susceptible to be able to introduce new business features faster or being able to replatform faster or add additional functionality faster.

The basics are simple. Make sure that you write your code using the best architectural principles of having your code modularized, making your code encapsulated, and having your code written with the appropriate abstraction layers. As mundane and as rudimentary this sounds, this is the basics. If your code is broken down into specific modules, you can easily remove modules, add modules. If you encapsulate specific functionality within your code in a coherent manner, again, you can add functionality, remove functionality with no issues.

Same thing with appropriate abstraction layers where people that are responsible for specific business functions, they write specific business functions, people that want to deal with data, they deal with data. So, using the right engineering principles will make things better in the future. Is it bulletproof? No. But is it going to make things easier as life goes on? Definitely.

When we're talking about AI, it's essentially the same thing. As long as your platforms and code is modular, and your data is clean, in other words, you don't have swamps, but rather clean data lakes, you'll definitely be able to utilize AI, whether it is for the actual software development lifecycle or to add features faster to your software. That will always help. If you're going to contaminate your environment with technical debt and bad data, things are going to be more complicated, and as much as AI going to be able to help you do things, it will be it will still be small.

With regards to embedding AI into actual products, that all depends on the use cases of the products. If you have a product that, for example, does some sort of a financial forecasting for financial industries, embedding AI where that AI can essentially take certain parameters from the real world and public data and maybe some proprietary data from that bank and do some predictive analysis and generate certain outcomes based on certain market conditions, of course. I mean, that that is the trend. That's what people are currently trying to do. And both generative AI and predictive AI will continue to help make products better and more intelligent. So, I definitely see people continuing to embed AI into products and it will be more and more. But, of course, you always have to take things with a grain of salt because no matter how good your prediction is, it still will not guarantee a particular outcome, right? Going back to the simple saying in stock investing, past results don't don't guarantee future outcomes. So, you still got to take everything with a grain of salt.

All right. Well, Alex, I think that's a that's a good one to wrap it up for this one. So, I would just like to thank you very much for taking the time and walking through a few of the questions and your point of view on them. I would just like, you know, to say it was really great on my end to hear you, to hear your answers, and to dive deep in, and maybe we hear each other again on another episode of the this podcast. Thank you very much.

Great. Thank you. Good to be here.