Decoding .NET8: Unveiling Cloud-Native Observability

The .NET programming language is taking cloud native deployment and observability seriously, and most notably with the recent announcement of .NET Aspire stack unveiled at the recent .NET Conf 2023. 

In the latest episode of OpenObservability Talks, we reviewed the journey to making .NET a “by default, out of the box observable platform,” as ASP.NET Core creator David Fowler put it. 

Fowler was my guest for this episode, and with him we dived into .NET Aspire and how it simplifies the complexities of cloud app development with capabilities around service discovery, observability, and resilience. We discussed the local developer experience, the path to developer observability, and what we can expect from the upcoming GA release of .NET8.

Out of the Box Observability with OpenTelemetry

The adoption of OpenTelemetry within Microsoft emerged as a critical facet of our discussion. While Fowler and his .NET team were pushing for OpenTelemetry’s seamless integration into the .NET ecosystem, they found allies within Microsoft in the Application Insights team. 

This team, in charge of Microsoft’s backend APM system, drove a strategic move to align Microsoft’s backend services with OpenTelemetry, ensuring a robust observability framework for teams building services within the .NET environment. Notably, the Microsoft team also features a distro of OpenTelemetry, dubbed Azure Monitor OpenTelemetry Distro, that automatically collects traces, metrics, logs, and exceptions across your application and its dependencies.

Fowler also highlighted Microsoft’s members in the board of OpenTelemetry, an open source project under the Cloud Native Computing Foundation, as a great source of alignment. He described this strategic collaboration with the board members, providing a direct channel to individuals actively working on the specification and components, which was particularly beneficial in determining adoption in early stages of the OpenTelemetry project. 

He shared insights into the challenges faced during the adoption, including a pivotal decision to adopt OpenTelemetry semantic conventions for HTTP as it became ratified. This ratification came late in the release cycle of .NET8, and making such a significant change was far from trivial, and they needed to take the leap of faith with OpenTelemetry, “And the team’s default nature culturally is to be very conservative. So like, ‘oh, we shouldn’t adopt it, it is the default answer. We should just not do it because what if it changes?’ And we bit the bullet.”

This decision to adopt the entire semantic conventions for HTTP was a bold move for Microsoft, demonstrating its commitment to embracing open standards and staying at the forefront of observability technologies.

.NET Aspire: Cloud Native Development Made Easy

As we delved into the conversation, Fowler provided a compelling overview of the .NET Aspire stack, emphasizing its role in simplifying cloud app development. 

For most developers, who are less familiar with the intricacies of cloud-native technologies, Aspire offers a quick start, as Fowler describes: “Oh, I got a container running for Redis and my projects and Postgres, and I can run locally, test, get discovery working between services.”

Admittedly, experienced developers and DevOps engineers, who are well versed with Kubernetes, Docker, Helm, Terraform, OpenTofu and the likes, might see less value in Aspire, and may even feel that it hides details from them, and are therefore not the main target audience.

When I first looked at Aspire, I saw elements that already exist in .NET, such as health checks, configuration, and dependency injection, and was wondering how Aspire maps to those. Fowler clarified that the Aspire stack isn’t an entirely new creation but rather a harmonious amalgamation of existing .NET elements. He clarified, “Aspire puts together things that are what we call Lego pieces or these atoms of .NET into this experience.” 

The stack’s focus on telemetry, health checks, and configurable frameworks empowers developers to build cloud-native applications efficiently.

Visualizing Telemetry in Local Dev Environments

Aspire stack doesn’t end with telemetry instrumentation and collection, but also provides a simple UI for telemetry visualization. Fowler says that Aspire has wired up an end-to-end experience where when you launch your app, the dashboard hosts an OTLP server and all the apps are configured to send telemetry to the dashboard “So you get this really immediate view of telemetry in your application with zero effort.”

Interestingly, the team decided to develop its own UI for Aspire, rather than leveraging the popular Prometheus & Grafana stack. Fowler shared the philosophy behind the decision: “When you’re doing development, you just want to see data. You don’t care about authoring queries, you don’t care about having alerting views or looking at percentiles. You just want to see the raw information.”

From Development to Production

After discussing the dev-test cycle, we shifted to discussing the transition from development to production. Fowler shared the current deployment pattern, where a local orchestrator generates a JSON manifest describing all resources and dependencies. Fowler mentioned tools like Azure Developer CLI (azd) for Azure and Aspirate (Aspir8) for Kubernetes, which were introduced to convert the Aspire manifest into deployable entities. 

Interestingly, Fowler also mentioned that early adopters of Aspire also expressed interest in using Aspire for production, whether the UI or even the orchestrator. While the team originally designated Aspire for the dev-test phases of software development, this yields an interesting evolution direction for the platform that the team is exploring.

Aspire vs. Dapr: Different Philosophies

To cap off our exploration, Fowler provided valuable insights into the distinction between Microsoft’s Dapr and the newly introduced Aspire stack. He emphasized that Dapr is an abstraction that engineers code against, run locally and then run in the Dapr cloud: “You use Dapr because you want to use the state store abstraction or you want to use the pub-sub abstraction.”

In contrast, Fowler emphasized that Aspire is not a cloud abstraction but a multi-cloud solution, that you can use whether you run on AWS, Azure or other, to make it easier: “We do not abstract storage, we don’t abstract pub-sub, we don’t abstract databases. You use the frameworks you love and know… We just add some stuff to those frameworks to make it do telemetry, health checks and make it more configurable.”

This is the proverbial difference between Platform as a Service (PaaS) and Platform Engineering: PaaS frameworks such as Heroku and Google AppEngine creates opinionated abstractions for the sake of simplicity, while Platform Engineering adapts to existing tech stacks without imposing a new paradigm, for the sake of flexibility.

Heading to GA of .NET8 and Aspire

Aspire was announced at .NETConf in November 2023, and has been progressing at an impressive rate since, with a monthly preview release and an ambitious roadmap. The upcoming Preview 4, according to Fowler, will “probably be the biggest release of the bunch because we are essentially taking some high level scenarios and making sure the end-to-end works.” 

And when is the GA expected? .NET8 is expected to reach general availability “before the summer (US time)” as Fowler shared, and with it, the GA release of the Aspire stack.

In the aftermath of this enriching conversation with David Fowler, it is evident that the .NET Aspire stack is not merely an evolution but a revolution in cloud-native development. The seamless integration of OpenTelemetry, the out-of-the-box telemetry instrumentation, the commitment to open specifications, coupled with .NET’s renowned integrated developer experience, all contribute to making .NET a truly cloud-native and observable platform. 

Want to learn more? Check out the OpenObservability Talks episode: Decoding .NET8: Unveiling Cloud-Native Observability.