Blog

Contoso loves the results of using a microservices architecture so far. The overall web application calls individual microservices to provide and manipulate data.

But as more services are added, the overall system becomes more complex to scale out and manage. Orchestrators can help.

What is an orchestrator?

An orchestrator is a tool that helps you manage, scale, and maintain a containerized application.

Using orchestrators for production-ready applications is essential if your application is based on microservices or is split across multiple containers. As noted earlier, in a microservice-based approach, each microservice owns its model and data. The microservice is autonomous from a development and deployment point of view. These kinds of systems are complex to scale out and manage. Therefore, to have a production-ready and scalable multi-container application, you absolutely need an orchestrator.

python programming training courses malaysia

The cloud drives today’s application development and IT systems management. Modern cloud applications need to be fast, agile, massively scalable, and reliable.

Using containers can help you deploy applications that meet all of those requirements. But putting an application into a container without following a strategic design pattern is like getting into a vehicle and hoping to find your way to a new city without using a map or GPS. You might end up at your destination, but the route probably won’t be direct or the most efficient.

A microservices architecture is useful in this scenario. Microservices give you an approach to software development and deployment that’s perfectly suited to the agility, scalability, and reliability requirements of modern cloud applications.

What is a microservices architecture?

In a microservices architecture, a large application is split up into a set of smaller services. Each service runs in its own process and communicates with other processes by using protocols like HTTP/HTTPS, WebSocket, or Advanced Message Queuing Protocol (AMQP). Each microservice implements a specific, end-to-end domain or business capability within a certain context boundary. Each microservice must be developed autonomously and must be independently deployable. Finally, each microservice should own its related domain data model and domain logic. Microservices can be based on different data storage technologies (SQL, NoSQL) and different programming languages.

red hat certified architect rhca malaysia

Creating microservices for a distributed environment presents a significant challenge. Cloud-hosted microservices often run in multiple containers in various regions. Implementing a solution that separates each service’s code from configuration eases the triaging of issues across all environments.

In this unit, explore how to integrate ASP.NET Core and Docker configuration features with Azure App Configuration to tackle this challenge in an effective way.

You’ll review the:

• ASP.NET Core configuration infrastructure.
• Kubernetes configuration abstraction—the ConfigMap.
• Azure App Configuration service.
• .NET Feature Management library.
• Feature flag components implemented in the app.

red hat certified engineer rhce malaysia

Your app is collecting telemetry data, and you now need a more comprehensive way to view it. In this unit, you’ll see how to view telemetry data in many different tools.

Monitor and view telemetry data

A common way to view telemetry data is to use Prometheus and Grafana together. Prometheus is an open-source monitoring system that collects metrics from your cloud-native app. Grafana is an open-source visualization tool that you use to create dashboards to view the metrics collected by Prometheus.

Prometheus

The first step is to add a Prometheus container, and configure it to collect data from each microservice in your app. You then add the Prometheus .NET client library to collect metrics from the app.

OpenTelemetry provides an exporter for Prometheus. You can add this exporter to your app by including the OpenTelemetry.Exporter.Prometheus.AspNetCore NuGet package. This package exports metrics to Prometheus in a format that it can understand. You replace the current console exporter with the Prometheus exporter.

You add the endpoints for all the microservices in your app. For example:

ymlCopy

global:
  scrape_interval: 1s

scrape_configs:
  - job_name: 'products'
    static_configs:
      - targets: ['backend:8080']
  - job_name: 'store'
    static_configs:
      - targets: ['frontend:8080']

mysql database training courses malaysia

Now that you understand the importance of observability, you’ll see how you can include it in your cloud-native application. You do this step by adding OpenTelemetry to your app.

Add OpenTelemetry to your app

.NET has a rich ecosystem of built-in observability tools that produce logging, metrics, and tracing data. You can use these tools to add observability to your cloud-native application. These libraries are:

• Logging: Microsoft.Extensions.Logging.ILogger
• Metrics: System.Diagnostics.Metrics.Meter
• Tracing: System.Diagnostics.Activity and System.Diagnostics.ActivitySource

red hat certified specialist in deployment and systems management malaysia

Before you dive into the code, let’s take a step back and talk about observability. You need to have tools that help you understand the internal state of a system based on what’s visible externally.

Why implement observability?

There are several reasons why observability is a crucial aspect of developing and improving cloud-native applications:

• Understanding system behavior: Observability provides insights into how your application is performing and where bottlenecks or errors occur.
• Debugging and troubleshooting: When issues arise, observability tools can provide detailed information about what was happening in the system at the time of the issue.
• Continuous improvement: Observability isn’t just for identifying and resolving issues—it’s also used for continuous improvement. By monitoring system performance over time, you can identify opportunities to optimize your code, improve system performance, and quantitatively improve the experience to your users.
• Proactive issue detection: With the right observability tools in place, you can often detect issues before your users even see them.

Implementing observability in cloud-native applications isn’t just a best practice, it’s a necessity for maintaining, optimizing, and continuously improving your applications. It empowers developers to deliver reliable, high-performing applications and provides the insights needed to drive informed decision-making.

The three pillars of observability

There are three main pillars of observability:

• Logs: Logs provide detailed records of events that occur within an application or system. In your app, you can use Microsoft.Extensions.Logging infrastructure to log events.
• Metrics: Metrics refer to numerical measurements and counters that provide insight into a system’s performance and health. Examples include request rates, response times, CPU/memory usage, and error rates. In your app, you might have specific measurements that you need to track.
• Distributed tracing: This process involves tracing a request as it propagates through all the microservices in your cloud-native app. Each service logs tracing data like request IDs that allow you to correlate events across services. Distributed tracing is also useful for debugging performance issues and errors in complex systems.

Together, these three pillars provide comprehensive observability into a system.

Sources of data for telemetry

In your company’s cloud-native app, there are several sources of telemetry you could choose to collect:

• Application logs: Applications generate logs that provide detailed information about its operation and errors if they occur. Logging is a rich source of telemetry data.
• Databases: Databases can provide telemetry data about the queries they process, execution times, and any errors that occur.
• HTTP requests and responses: The HTTP requests and responses between your microservices provide rich and valuable telemetry data. This data includes the request and response headers, body content, status codes, and timing information.
• Client-side performance data: In cloud-native apps with a front end, you can collect the client-side performance data. This data might include page hits, load times, and UI interaction times.
• Infrastructure metrics: If your application is hosted in a cloud environment, you can collect infrastructure metrics like CPU usage, memory usage, network traffic, and disk I/O operations.

red hat certified specialist in red hat enterprise linux diagnostics and troubleshooting malaysia

Compliance departments need to be able to review code and compliance reports to ensure that the application is compliant with the company’s policies. The .NET compliance framework provides a way to generate reports that show the compliance status of the application.

What is a compliance report?

A compliance report can be generated at compile time. The .NET compliance framework generates a JSON file that contains details on the data classifications and redaction methods used in the application.

JSONCopy

{
    "Name": "DataEntities",
    "Types": [
        {
            "Name": "DataEntities.Order",
            "Members": [
                {
                    "Name": "CustomerAddress",
                    "Type": "string",
                    "File": "C:\\Developer\\mslearn-dotnet-cloudnative\\dotnet-compliance\\eShopLite\\DataEntities\\Order.cs",
                    "Line": "25",
                    "Classifications": [
                        {
                            "Name": "EUIIData"
                        }
                    ]
                },
                {
                    "Name": "CustomerName",
                    "Type": "string",
                    "File": "C:\\Developer\\mslearn-dotnet-cloudnative\\dotnet-compliance\\eShopLite\\DataEntities\\Order.cs",
                    "Line": "21",
                    "Classifications": [
                        {
                            "Name": "EUIIData"
                        }
                    ]
                },
    ...

The report above is an example from the eShopLite.DataEntities project. It shows that the Order class has two properties that are classified as EUIIData.

JSONCopy

{
    "Name": "Store",
    "Types": [
        {
            "Name": "Store.Services.Log",
            "Logging Methods": [
                {
                    "Name": "LogOrders",
                    "Parameters": [
                        {
                            "Name": "logger",
                            "Type": "Microsoft.Extensions.Logging.ILogger",
                            "File": "C:\\Developer\\mslearn-dotnet-cloudnative\\dotnet-compliance\\eShopLite\\Store\\Services\\ProductService.cs",
                            "Line": "103"
                        },
                        {
                            "Name": "order",
                            "Type": "DataEntities.Order",
                            "File": "C:\\Developer\\mslearn-dotnet-cloudnative\\dotnet-compliance\\eShopLite\\Store\\Services\\ProductService.cs",
                            "Line": "103"
                        }
                    ]
                }
            ]
        }
    ]
}

The report above is an example from the eShopLite.Store project. It shows that the LogOrders method in the ProductService class takes an Order object as a parameter for logging.

red hat certified specialist in linux performance tuning malaysia

The following example is for a Blazor WebAssembly app. The process is similar for other types of apps, but the code is slightly different depending on how the dependency injection container is configured.

In the program.cs file, add the following dependencies:

C#Copy

using Microsoft.Extensions.Compliance.Classification;
using Microsoft.Extensions.Compliance.Redaction;

The above packages allow you to then add the redaction service to the dependency injection container with this code:

C#Copy

builder.Services.AddRedaction();

Choose which redaction implementation to use for each type of classified data

The AddRedactor method can include a RedactorOptions parameter. The parameter allows you to specify which redaction implementation to use for each data taxonomy.

For example, the following code specifies that the HMACSHA256Redactor should be used for EUII data.

C#Copy

builder.Services.AddRedaction(configure =>
{
    // Configure to use the HMAC redactor
    configure.SetHmacRedactor(configureHmac =>
    {
        // This key should be fetched from keyvault or some other secure store.
        configureHmac.Key = "thisisadummykeythatshouldbereplacedwithakeyfromakeystore";
        // Some discriminator to differentiate between different deployments of a service.
        configureHmac.KeyId = 1;

    }, new DataClassificationSet(DataClassifications.EUIIDataClassification));
});

red hat certified specialist in server hardening malaysia

Redaction within applications is most commonly done on log messages and telemetry. It can also be used in other scenarios like redacting dimensions in Metrics, or header data in middleware.

The .NET logging framework provides a simple way to redact data in log messages. The Microsoft.Extensions.Compliance.Abstractions package enhances logging to include a Redactor class that redacts data.

What is redaction?

Redaction is the process of removing sensitive information from a message. For example, you might want to redact a user’s name from a log message. Or you might want to redact a user’s IP address from a telemetry event.

The most simple redaction is to erase the value, and return an empty string for a variable. This behavior happens by default because the ErasingRedactor is default fallback redactor. Microsoft includes a HMACSHA256Redactor class that can be used to redact data using a hash function. The HMAC redaction is useful if you want to redact data, but still be able to correlate log messages across multiple log statements. The last option is to provide your own redaction function, which is useful if you want to redact data using a custom algorithm.

For example, you want to make it clearer in the logs that a value is redacted by replacing it with *****.

How to redact data in a cloud-native application

Your organizations cloud-native app could be writing logs and creating telemetry in multiple projects. For example, it could be writing logs from the database service, the web app, or any other API’s it uses. Depending on the type of logging, you’ll need to add the redaction service to each one.

There are four steps you need to take to enable redaction in your app:

1. Add the Microsoft.Extensions.Compliance.Redaction NuGet package to each project.
2. Add the redaction service to the dependency injection container.
3. Choose which redaction implementation to use for each type of classified data.
4. Enable redaction in the logging framework.

red hat enterprise linux rhel training courses malaysia

Microsoft has added a new extension to .NET that makes it easy to implement data classification. The Microsoft.Extensions.Compliance.Classification extension enables you to define DataClassification and DataClassificationAttribute properties.

To use the extension in your solution, add the Microsoft.Extensions.Compliance.Redaction NuGet package to your project.

For example the code to create the above taxonomies could look like this:

C#Copy

using Microsoft.Extensions.Compliance.DataClassification;

public static DataClassification EUIIDataClassification {get;} = new DataClassification("EUIIDataTaxonomy", "EUIIData");

public static DataClassification EUPDataClassification {get;} = new DataClassification("EUPDataTaxonomy", "EUPData");

public class EUIIDataAttribute : DataClassificationAttribute
{
    public EUIIDataAttribute() : base(DataClassifications.EUIIDataClassification) { }
}

public class EUPDataAttribute : DataClassificationAttribute
{
    public EUPDataAttribute() : base(DataClassifications.EUPDataClassification) { }
}

With your taxonomies defined, you can now annotate your data types with the appropriate attribute. For example:

C#Copy

public class User
{
    [EUIIData]
    public string Name { get; set; }

    [EUIIData]
    public string Address { get; set; }

    [EUPData]
    public string UserId { get; set; }
}

red hat linux administration training courses malaysia