Blog

GitHub Copilot is powered by Large Language Models (LLMs) to assist you in writing code seamlessly. In this unit, we focus on understanding the integration and impact of LLMs in GitHub Copilot. Let’s review the following topics:

• What are LLMs?
• The role of LLMs in GitHub Copilot and prompting
• Fine-tuning LLMs
• LoRA fine-tuning

What are LLMs?

Large Language Models (LLMs) are artificial intelligence models designed and trained to understand, generate, and manipulate human language. These models are ingrained with the capability to handle a broad range of tasks involving text, thanks to the extensive amount of text data they’re trained on. Here are some core aspects to understand about LLMs:

Volume of training data

LLMs are exposed to vast amounts of text from diverse sources. This exposure equips them with a broad understanding of language, context, and intricacies involved in various forms of communication.

Contextual understanding

They excel in generating contextually relevant and coherent text. Their ability to understand context allows them to provide meaningful contributions, be it completing sentences, paragraphs, or even generating whole documents that are contextually apt.

Machine learning and AI integration

LLMs are grounded in machine learning and artificial intelligence principles. They’re neural networks with millions, or even billions, of parameters that are fine-tuned during the training process to understand and predict text effectively.

Versatility

These models aren’t limited to a specific type of text or language. They can be tailored and fine-tuned to perform specialized tasks, making them highly versatile and applicable across various domains and languages.

ccie certification training courses malaysia

In this unit, we’ll cover how GitHub Copilot handles data for different environments, features and configurations.

Data handling for GitHub Copilot code suggestions

GitHub Copilot in the code editor does not retain any prompts like code or other context used for the purposes of providing suggestions to train the foundational models. It discards the prompts once a suggestion is returned.

GitHub Copilot Individual subscribers can opt-out of sharing their prompts with GitHub which will otherwise be used to finetune GitHub’s foundational model.

Data handling for GitHub Copilot chat

GitHub Copilot Chat operates as an interactive platform, allowing developers to engage in conversational interactions with the AI assistant to receive coding assistance. Here are the steps that it carries out which might be distinct from other features like code completion:

• Formatting: Copilot meticulously formats the generated response for optimal presentation within the chat interface. It highlights code snippets to improve readability and may include options for direct integration into your code. Copilot showcases the formatted response in the Copilot Chat window within the IDE, allowing you to easily review and interact with the provided information.
• User engagement: You can actively engage with the response by asking follow-up questions, requesting clarifications, or providing additional input. The chat interface maintains a conversation history to facilitate contextual understanding in subsequent interactions.
• Data retention: For Copilot Chat used outside the code editor, GitHub typically retains prompts, suggestions, and supporting context for 28 days. Specific retention policies for Copilot Chat within the code editor may vary.

The same goes for CLI, Mobile, and GitHub Copilot Chat on GitHub.com.

Prompt types supported by GitHub Copilot Chat

GitHub Copilot Chat processes a wide range of coding-related prompts, demonstrating its versatility as a conversational coding assistant. Here are some common input types:

• Direct Questions: You can ask specific questions about coding concepts, libraries, or troubleshooting issues. For example, “How do I implement a quick sort algorithm in Python?” or “Why is my React component not rendering?”
• Code-Related Requests: You can request code generation, modification, or explanation. Examples include “Write a function to calculate factorial,” “Fix this error in my code,” or “Explain this code snippet.”
• Open-Ended Queries: You can explore coding concepts or seek general guidance by asking open-ended questions like “What are the best practices for writing clean code?” or “How can I improve the performance of my Python application?”
• Contextual Prompts: You can provide code snippets or describe specific coding scenarios to seek tailored assistance. For instance, “Here’s a part of my code, can you suggest improvements?” or “I’m building a web application, can you help me with the authentication flow?”

microsoft 365 certification training courses malaysia

In this unit, we’ll cover:

• What is prompt engineering?
• Foundations of prompt engineering
• Best practices in prompt engineering
• How Copilot learns from your prompts

What is prompt engineering?

Prompt engineering is the process of crafting clear instructions to guide AI systems, like GitHub Copilot, to generate context-appropriate code tailored to your project’s specific needs. This ensures the code is syntactically, functionally, and contextually correct.

Now that you know what prompt engineering is, let’s learn about some of its principles.

Principles of prompt engineering

Before we explore specific strategies, let’s first understand the basic principles of prompt engineering, summed up in the 4 Ss below. These core rules are the basis for creating effective prompts.

• Single: Always focus your prompt on a single, well-defined task or question. This clarity is crucial for eliciting accurate and useful responses from Copilot.
• Specific: Ensure that your instructions are explicit and detailed. Specificity leads to more applicable and precise code suggestions.
• Short: While being specific, keep prompts concise and to the point. This balance ensures clarity without overloading Copilot or complicating the interaction.
• Surround: Utilize descriptive filenames and keep related files open. This provides Copilot with rich context, leading to more tailored code suggestions.

These core principles lay the foundation for crafting efficient and effective prompts. Keeping the 4 Ss in mind, let’s dive deeper into advanced best practices that ensure each interaction with GitHub Copilot is optimized.

Best practices in prompt engineering

The following advanced practices, based on the 4 Ss, refine and enhance your engagement with Copilot, ensuring that the generated code isn’t only accurate but perfectly aligned with your project’s specific needs and contexts.

Provide enough clarity

Building on the ‘Single’ and ‘Specific’ principles, always aim for explicitness in your prompts. For instance, a prompt like “Write a Python function to filter and return even numbers from a given list” is both single-focused and specific.

Provide enough context with details

Enrich Copilot’s understanding with context, following the ‘Surround’ principle. The more contextual information provided, the more fitting the generated code suggestions are. For example, by adding some comments at the top of your code to give more details to what you want, you can give more context to Copilot to understand your prompt, and provide better code suggestions.

oracle cloud infrastructure training courses malaysia

Microsoft Security Copilot integrates with various sources, including Microsoft’s own security products, non-Microsoft vendors, open-source intelligence feeds, websites, and knowledge bases to generate guidance that’s specific to your organization.

One of the mechanisms by which Copilot integrates to these various sources is through plugins. Plugins extend Copilot’s capabilities. In this unit, you’ll explore the Microsoft plugins.

Microsoft plugins

Microsoft plugins give Copilot access to information and capabilities from within your organization’s Microsoft products. The image that follows shows only a subset of the available Microsoft plugins and the order in which the plugins are listed may vary from what is displayed in the product.

If a Copilot owner has restricted plugin access, then those plugins that have been set to restricted will show greyed out and restricted.

• Plugins
• Restricted plugins

Generally speaking, Microsoft plugins in Copilot use the OBO (on behalf of) model – meaning that Copilot knows that a customer has licenses to specific products and is automatically signed into those products. Copilot can then access the specific products when the plugin is enabled and, where applicable, parameters are configured. Some Microsoft plugins that require setup, as noted by the settings icon or the setup button, may include configurable parameters that are used for authentication in-lieu of the OBO model.

To view the system capabilities supported by the enabled plugins, you select the prompt icon located in the prompt bar and select “See all system capabilities.” System capabilities are specific, single prompts that you can use in Copilot. Selecting a system capability typically requires more input to get a useful response, but Copilot provides that guidance.

The sections that follow provide brief descriptions for many, but not all, of the available Microsoft plugins. Microsoft Security Copilot is continually adding support for Microsoft products.

Azure Firewall (Preview)

Azure Firewall is a cloud-native and intelligent network firewall security service that provides best of breed threat protection for your cloud workloads running in Azure. It’s a fully stateful firewall as a service with built-in high availability and unrestricted cloud scalability.

The Azure Firewall integration with Copilot helps analysts perform detailed investigations of the malicious traffic intercepted by the intrusion detection and prevention system (IDPS) and/or the threat intelligence capabilities of the firewalls across their environment.

To use the Azure Firewall integration with Copilot:

• The Azure Firewalls to be used with Security Copilot must be configured with resource specific structured logs for IDPS and these logs must be sent to a Log Analytics workspace.
• The users using the Azure Firewall plugin in Security Copilot must have the appropriate Azure role-based access control (RBAC) roles to access the Firewall and associated Log Analytics workspace.
• The Azure Firewall plugin in Security Copilot must be turned on.

oracle database training courses malaysia

A session is a particular conversation within Copilot and consists of one or more prompts. A session may be initiated through prompts you enter directly in the prompt bar, by running a promptbook, or a prompt suggestion.

Copilot has features that are common across all sessions and the individual prompts that make up a session, including:

• The process log
• Actions you can take a prompt and its response
• Prompt feedback
• The pin board

Process log

For every prompt Copilot runs, Copilot generates a process log that is visible to the user. The user can see what capability is used to generate the response. This is important because it enables the user to determine whether the response was generated from a trusted source. In the screenshot that follows, the process log shows that Copilot chose the Incident Analysis capability. The process log also shows that the final output went through safety checks, which is part of Microsoft’s commitment to responsible AI.

Action you can take on prompt and its response

There’s a consistent set of actions available for every prompt/response pair in the Copilot standalone experience. Actions include:

• Pin an individual prompt and its corresponding response to the pin board by selecting the pin icon.
• Edit a prompt.
• Rerun a prompt.
• Delete a prompt.
• Export a prompt.
• Copy a response.
• Provide feedback on a response

oracle e business suite training courses malaysia

The Copy Data activity is one of the most common uses of a data pipeline. Many pipelines consist of a single Copy Data activity that is used to ingest data from an external source into a lakehouse file or table.

You can also combine the Copy Data activity with other activities to create a repeatable data ingestion process – for example by using a Delete data activity to remove existing data, a Copy Data activity to replace the deleted data with a file containing data from an external source, and a Notebook activity to run Spark code that transforms the data in the file and loads it into a table.

The Copy Data tool

When you add a Copy Data activity to a pipeline, a graphical tool takes you through the steps required to configure the data source and destination for the copy operation. A wide range of source connections is supported, making it possible to ingest data from most common sources. In OneLake, this includes support for lakehouse, warehouse, SQL Database, and others.

Copy Data activity settings

After you’ve added a Copy Data activity to a pipeline, you can select it in the pipeline canvas and edit its settings in the pane underneath.

When to use the Copy Data activity

Use the Copy Data activity when you need to copy data directly between a supported source and destination without applying any transformations, or when you want to import the raw data and apply transformations in later pipeline activities.

oracle linux administration training courses malaysia

Pipelines in Microsoft Fabric encapsulate a sequence of activities that perform data movement and processing tasks. You can use a pipeline to define data transfer and transformation activities, and orchestrate these activities through control flow activities that manage branching, looping, and other typical processing logic. The graphical pipeline canvas in the Fabric user interface enables you to build complex pipelines with minimal or no coding required.

Core pipeline concepts

Before building pipelines in Microsoft Fabric, you should understand a few core concepts.

Activities

Activities are the executable tasks in a pipeline. You can define a flow of activities by connecting them in a sequence. The outcome of a particular activity (success, failure, or completion) can be used to direct the flow to the next activity in the sequence.

There are two broad categories of activity in a pipeline.

• Data transformation activities – activities that encapsulate data transfer operations, including simple Copy Data activities that extract data from a source and load it to a destination, and more complex Data Flow activities that encapsulate dataflows (Gen2) that apply transformations to the data as it is transferred. Other data transformation activities include Notebook activities to run a Spark notebook, Stored procedure activities to run SQL code, Delete data activities to delete existing data, and others. In OneLake, you can configure the destination to a lakehouse, warehouse, SQL database, or other options.
• Control flow activities – activities that you can use to implement loops, conditional branching, or manage variable and parameter values. The wide range of control flow activities enables you to implement complex pipeline logic to orchestrate data ingestion and transformation flow.

oracle peoplesoft training courses malaysia

To implement infrastructure-based resiliency, you can use a service mesh. Aside from resiliency without changing code, a service mesh provides traffic management, policy, security, strong identity, and observability. Your app is decoupled from these operational capabilities, which are moved to the infrastructure layer. Architecturally speaking, a service mesh is composed of two components: a control plane and a data plane.

The control plane component has many components that support managing the service mesh. The components inventory typically includes:

• A management interface, which could be a UI or an API.
• Rules and policy definitions that define how the service mesh should implement specific capabilities.
• Security management for things like strong identity and certificates for mTLS.
• Metrics or observability to collect and aggregate metrics and telemetry from the apps.

oracle siebel crm training courses malaysia

The resiliency features of .NET are built upon the Polly project and made available through Microsoft.Extensions. You can add a standard resilience strategy that uses sensible defaults by adding a single line of code to your app.

Add resilience to your app

To add resilience to an app built using a microservices architecture, using HTTP requests between individual services, take these steps:

1. Add the Microsoft.Extensions.Http.Resilience package to your project.
2. Add a resilience handler to your HttpClient service calls.
3. Configure the resilience strategy.

Add the NuGet package to your project

Run the following command to add the resiliency NuGet package:

.NET CLICopy

dotnet add package Microsoft.Extensions.Http.Resilience

Running this command from the terminal in the apps project folder will add the package reference to the project file.

Then add the following using statement in your application’s startup class :

C#Copy

using Microsoft.Extensions.Http.Resilience;

Add a resilience strategy

You can now add a standard resilience strategy to your HttpClient service. .NET provides this out-of-the-box configuration combining a number of strategies.

The request handler goes through each of these strategies in order form left to right:

• Total request timeout strategy: This sets a total amount of time that the request can take. You can think of this as setting the upper time limit for all the other strategies.
• Retry strategy: This strategy controls the options on number of retries, backoff, and jitter. These options can’t exceed the total timeout set in the previous strategy.
• Circuit breaker strategy: This strategy opens the circuit if the failure ratio exceeds the threshold.
• Attempt timeout strategy: This strategy sets a timeout for each individual request. If the request takes longer than this time, then an exception is thrown.

You can add this standard strategy, with all the default values by adding this extension method:

C#Copy

.AddStandardResilienceHandler();

For example if you have declared a WebApplication, and you want to add a resilience strategy to the HttpClient service use this code:

C#Copy

builder.Services.AddHttpClient<ServiceBeingCalled>(httpClient =>
{
    httpClient.BaseAddress = new Uri("https://service.endpoint/");
}).AddStandardResilienceHandler();

The first line of the preceding code adds a standard resilience handler to the HTTPClient. This will use all the default settings for the retry and circuit breaker strategies.

php programming training courses malaysia

A Retry strategy is exactly what the name implies. The request is retried after a short wait if an error response is received. The wait time increases with each retry. The increase can be linear or exponential.

After the maximum number of retries is reached, the strategy gives up and throws an exception. From the user’s perspective, the app usually takes longer to complete some operations. The app might also take some time before informing the user that it couldn’t complete the operation.

Circuit Breaker strategy

A Circuit Breaker strategy gives a target service a break after a repeated number of failures by pausing trying to communicate with it. The service could be experiencing a serious problem and be temporarily unable to respond. After a defined number of consecutive failures, the connection attempts are paused, opening the circuit. During this wait, additional operations on the target service fail immediately without even trying to connect the service. After the wait time has elapsed, the operation is attempted again. If the service successfully responds, the circuit is closed and the system goes back to normal.

Infrastructure-based resiliency

To implement infrastructure-based resiliency, you can use a service mesh. Aside from resiliency without changing code, a service mesh provides traffic management, policy, security, strong identity, and observability. Your app is decoupled from these operational capabilities, which are moved to the infrastructure layer.

Comparison to code-based approaches

An infrastructure-based resiliency approach can use a metrics-based view that allows it to adapt dynamically to cluster conditions in real time. This approach adds another dimension to managing the cluster but doesn’t add any code.

power bi training courses malaysia