Blog

Each organization has their own guiding principles, but ultimately these principles need to be part of a larger responsible AI strategy to be effective. This strategy should encompass how your organization brings these principles to life both within your organization and beyond.

We recommend establishing a governance system that is tailored to your organization’s unique characteristics, culture, guiding principles, and level of engagement with AI. The tasks of the board should include designing responsible AI policies and measures; attending they’re being followed, and ensuring compliance.

To help your organization get started, we have provided an overview of three common governance approaches: hiring a Chief Ethics Officer, establishing an ethics office, and forming an ethics committee. The first approach is centralized, and the others are decentralized. All of them have their benefits, but we recommend combining them in a hybrid approach. A governance system that reports to the board of directors and has financial support, human resources, and authority is more likely to create real change across an organization.

design

To build trust, it’s critical that AI systems operate reliably, safely, and consistently under normal circumstances and in unexpected conditions. These systems should be able to operate as they were originally designed, respond safely to unanticipated conditions, and resist harmful manipulation. It’s also important to be able to verify that these systems are behaving as intended under actual operating conditions. How they behave and the variety of conditions they can handle reliably and safely largely reflects the range of situations and circumstances that developers anticipate during design and testing.

To ensure reliability and safety in your AI system, you should:

• Develop processes for auditing AI systems to evaluate the quality and suitability of data and models, monitor ongoing performance, and verify that systems are behaving as intended based on established performance measures.
• Provide detailed explanation of system operation including design specifications, information about training data, training failures that occurred and potential inadequacies with training data, and the inferences and significant predictions generated.
• Design for unintended circumstances such as accidental system interactions, the introduction of malicious data, or cyberattacks.
• Involve domain experts in the design and implementation processes, especially when using AI to help make consequential decisions about people.
• Conduct rigorous testing during AI system development and deployment to ensure that systems can respond safely to unanticipated circumstances, don’t have unexpected performance failures, and don’t evolve in unexpected ways. AI systems involved in high-stakes scenarios that affect human safety or large populations should be tested both in lab and real-world scenarios.
• Evaluate when and how an AI system should seek human input for impactful decisions or during critical situations. Consider how an AI system should transfer control to a human in a manner that is meaningful and intelligible. Design AI systems to ensure humans have the necessary level of input on highly impactful decisions.
• Develop a robust feedback mechanism for users to report performance issues so that you can resolve them quickly.

Privacy and security

As AI becomes more prevalent, protecting privacy and securing important personal and business information is becoming more critical and complex. With AI, privacy and data security issues require especially close attention because access to data is essential for AI systems to make accurate and informed predictions and decisions about people.

To ensure privacy and security in your AI system, you should:

• Comply with relevant data protection, privacy, and transparency laws by investing resources in developing compliance technologies and processes or working with a technology leader during the development of AI systems. Develop processes to continually check that the AI systems are satisfying all aspects of these laws.
• Design AI systems to maintain the integrity of personal data so that they can only use personal data during the time it’s required and for the defined purposes that have been shared with customers. Delete inadvertently collected personal data or data that is no longer relevant to the defined purpose.
• Protect AI systems from bad actors by designing AI systems in accordance with secure development and operations foundations, using role-based access, and protecting personal and confidential data that is transferred to third parties. Design AI systems to identify abnormal behaviors and to prevent manipulation and malicious attacks.
• Design AI systems with appropriate controls for customers to make choices about how and why their data is collected and used.
• Ensure your AI system maintains anonymity by taking into account how the system removes personal identification from data.
• Conduct privacy and security reviews for all AI systems.
• Research and implement industry best practices for tracking relevant information about customer data, accessing and using that data, and auditing access and use.

sample page

In the last unit, we discussed some of the societal implications of AI. We touched on the responsibility of businesses, governments, NGOs, and academic researchers to anticipate and mitigate unintended consequences of AI technology. As organizations consider these responsibilities, more are creating internal policies and practices to guide their AI efforts.

At Microsoft, we’ve recognized six principles that we believe should guide AI development and use: fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability. For us, these principles are the cornerstone of a responsible and trustworthy approach to AI, especially as intelligent technology becomes more prevalent in the products and services we use every day.

Fairness

AI systems should treat everyone fairly and avoid affecting similarly situated groups of people in different ways. For example, when AI systems provide guidance on medical treatment, loan applications, or employment, they should make the same recommendations to everyone with similar symptoms, financial circumstances, or professional qualifications.

To ensure fairness in your AI system, you should:

• Understand the scope, spirit, and potential uses of the AI system by asking questions such as, how is the system intended to work? Who is the system designed to work for? Will the system work for everyone equally? How can it harm others?
• Attract a diverse pool of talent. Ensure the design team reflects the world in which we live by including team members that have different backgrounds, experiences, education, and perspectives.
• Identify bias in datasets by evaluating where the data came from, understanding how it was organized, and testing to ensure it’s represented. Bias can be introduced at every stage in creation, from collection to modeling to operation. The Responsible AI Dashboard, available at the Resources section, includes a feature to help with this task.
• Identify bias in machine learning algorithms by applying tools and techniques that improve the transparency and intelligibility of models. Users should actively identify and remove bias in machine learning algorithms.
• Leverage human review and domain expertise. Train employees to understand the meaning and implications of AI results, especially when AI is used to inform consequential decisions about people. Decisions that use AI should always be paired with human review. Include relevant subject matter experts in the design process and in deployment decisions. An example would be including a consumer credit subject matter expert for a credit scoring AI system. You should use AI as a copilot, that is, an assisting tool that helps you do your job better and faster but requires some degree of supervising.
• Research and employ best practices, analytical techniques, and tools from other institutions and enterprises to help detect, prevent, and address bias in AI systems.

vmware vsphere install configure manage v8 icm malaysia

It’s important to recognize that as new intelligent technology emerges and proliferates throughout society, with its benefits come unintended and unforeseen consequences. Some of these consequences have significant ethical ramifications and the potential to cause serious harm. While organizations can’t predict the future yet, it’s our responsibility to make a concerted effort to anticipate and mitigate the unintended consequences of the technology we release into the world through deliberate planning and continual oversight.

Threats

Each breakthrough in AI technologies brings a new reminder of our shared responsibility. For example, in 2016, Microsoft released a chatbot on X called Tay, which could learn from interactions with X users. The goal was to enable the chatbot to better replicate human communication and personality traits. However, within 24 hours, users realized that the chatbot could learn from bigoted rhetoric, and turned the chatbot into a vehicle for hate speech. This experience is one example of why we must consider human threats when designing AI systems.

Novel threats require a constant evolution in our approach to responsible AI. For example, because generative AI enables people to create or edit videos, images, or audio files so credibly that they look real, media authenticity is harder to verify. In response, Microsoft is teaming with other technology and news stakeholders to develop technical standards to address deepfake-related manipulation.

ccna certification training courses malaysia

AI is the defining technology of our time. It’s already enabling faster and more profound progress in nearly every field of human endeavor and helping to address some of society’s most daunting challenges. For example, AI can help people with visual disabilities understand images by generating descriptive text for images. In another example, AI can help farmers produce enough food for the growing global population.

At Microsoft, we believe that the computational intelligence of AI should be used to amplify the innate creativity and ingenuity of humans. Our vision for AI is to empower every developer to innovate, empower organizations to transform industries, and empower people to transform society.

Societal implications of AI

As with all great technological innovations in the past, the use of AI technology has broad impacts on society, raising complex and challenging questions about the future we want to see. AI has implications on decision-making across industries, data security and privacy, and the skills people need to succeed in the workplace. As we look to this future, we must ask ourselves:

• How do we design, build, and use AI systems that create a positive impact on individuals and society?
• How can we best prepare workers for the effects of AI?
• How can we attain the benefits of AI while respecting privacy?

comptia training courses malaysia

Security and monitoring are critical aspects of managing your data warehouse.

Security

Data warehouse security is important to protect your data from unauthorized access. Fabric provides a number of security features to help you secure your data warehouse. These include:

• Role-based access control (RBAC) to control access to the warehouse and its data.
• TLS encryption to secure the communication between the warehouse and the client applications.
• Azure Storage Service Encryption to protect the data in transit and at rest.
• Azure Monitor and Azure Log Analytics to monitor the warehouse activity and audit the access to the data.
• Multifactor authentication (MFA) to add an extra layer of security to user accounts.
• Microsoft Entra ID integration to manage the user identities and access to the warehouse.

Workspace permissions

Data in Fabric is organized into workspaces, which are used to control access and manage the lifecycle of data and services. Appropriate workspace roles are the first line of defense in securing your data warehouse.

In addition to workspace roles, you can grant item permissions and access through SQL.

exchange server certification training courses malaysia

Now that you know how to implement a data warehouse in Fabric, let’s prepare the data for analytics.

There are two ways to query data from your data warehouse. The Visual query editor provides a no-code, drag-and-drop experience to create your queries. If you’re comfortable with T-SQL, you may prefer to use the SQL query editor to write your queries. In both cases, you can create tables, views, and stored procedures to query data in the data warehouse and Lakehouse.

There’s also a SQL analytics endpoint, where you can connect from any tool.

Query data using the SQL query editor

The SQL query editor provides a query experience that includes intellisense, code completion, syntax highlighting, client-side parsing, and validation. If you’ve written T-SQL in SQL Server Management Studio (SSMS) or Azure Data Studio (ADS), you’ll find it familiar.

dynamics 365 training courses malaysia

Fabric’s Lakehouse is a collection of files, folders, tables, and shortcuts that act like a database over a data lake. It’s used by the Spark engine and SQL engine for big data processing and has features for ACID transactions when using the open-source Delta formatted tables.

Fabric’s data warehouse experience allows you to transition from the lake view of the Lakehouse (which supports data engineering and Apache Spark) to the SQL experiences that a traditional data warehouse would provide. The Lakehouse gives you the ability to read tables and use the SQL analytics endpoint, whereas the data warehouse enables you to manipulate the data.

In the data warehouse experience, you’ll model data using tables and views, run T-SQL to query data across the data warehouse and Lakehouse, use T-SQL to perform DML operations on data inside the data warehouse, and serve reporting layers like Power BI.

Now that you understand the basic architectural principles for a relational data warehouse schema, let’s explore how to create a data warehouse.

dynamics 365 supply chain training courses malaysia

The process of building a modern data warehouse typically consists of:

• Data ingestion – moving data from source systems into a data warehouse.
• Data storage – storing the data in a format that is optimized for analytics.
• Data processing – transforming the data into a format that is ready for consumption by analytical tools.
• Data analysis and delivery – analyzing the data to gain insights and delivering those insights to the business.

Microsoft Fabric enables data engineers and analysts to ingest, store, transform, and visualize data all in one tool with both a low-code and traditional experience.

Understand Fabric’s data warehouse experience

Fabric’s data warehouse is a relational data warehouse that supports the full transactional T-SQL capabilities you’d expect from an enterprise data warehouse. It’s a fully managed, scalable, and highly available data warehouse that can be used to store and query data in the Lakehouse. Using the data warehouse, you’re fully in control of creating tables, loading, transforming, and querying data using either the Fabric portal or T-SQL commands. You can use SQL to query and analyze the data, or use Spark to process the data and create machine learning models.

Data warehouses in Fabric facilitate collaboration between data engineers and data analysts, working together in the same experience. Data engineers build a relational layer on top of data in the Lakehouse, where analysts can use T-SQL and Power BI to explore the data.

Design a data warehouse

Like all relational databases, Fabric’s data warehouse contains tables to store your data for analytics later. Most commonly, these tables are organized in a schema that is optimized for multidimensional modeling. In this approach, numerical data related to events (e.g. sales orders) are grouped by different attributes (e.g. date, customer, store). For instance, you can analyze the total amount paid for sales orders that occurred on a specific date or at a particular store.

Tables in a data warehouse

Tables in a data warehouse are typically organized in a way that supports efficient and effective analysis of large amounts of data. This organization is often referred to as dimensional modeling, which involves structuring tables into fact tables and dimension tables.

Fact tables contain the numerical data that you want to analyze. Fact tables typically have a large number of rows and are the primary source of data for analysis. For example, a fact table might contain the total amount paid for sales orders that occurred on a specific date or at a particular store.

Dimension tables contain descriptive information about the data in the fact tables. Dimension tables typically have a small number of rows and are used to provide context for the data in the fact tables. For example, a dimension table might contain information about the customers who placed sales orders.

In addition to attribute columns, a dimension table contains a unique key column that uniquely identifies each row in the table. In fact, it’s common for a dimension table to include two key columns:

• A surrogate key is a unique identifier for each row in the dimension table. It’s often an integer value that is automatically generated by the database management system when a new row is inserted into the table.
• An alternate key is often a natural or business key that identifies a specific instance of an entity in the transactional source system – such as a product code or a customer ID.

You need both surrogate and alternate keys in a data warehouse, because they serve different purposes. Surrogate keys are specific to the data warehouse and help to maintain consistency and accuracy in the data. Alternate keys on the other hand are specific to the source system and help to maintain traceability between the data warehouse and the source system.

Special types of dimension tables

Special types of dimensions provide additional context and enable more comprehensive data analysis.

Time dimensions provide information about the time period in which an event occurred. This table enables data analysts to aggregate data over temporal intervals. For example, a time dimension might include columns for the year, quarter, month, and day in which a sales order was placed.

Slowly changing dimensions are dimension tables that track changes to dimension attributes over time, like changes to a customer’s address or a product’s price. They’re significant in a data warehouse because they enable users to analyze and understand changes to data over time. Slowly changing dimensions ensure that data stays up-to-date and accurate, which is imperative to making good business decisions.

dynamics 365 sales training courses malaysia

Concurrent orchestration lets multiple agents work on the same task at the same time. Each agent handles the task independently, and then their outputs are gathered and combined. This method works especially well when you want diverse approaches or solutions, like during brainstorming, group decision-making, or voting.

This pattern is useful when you need different approaches or ideas to solve the same problem. Instead of having agents work one after another, they all work at the same time. This speeds up the process and covers the problem from many angles.

Usually, the results from each agent are combined to create a final answer, but this isn’t always necessary. Each agent can also produce its own separate result, like calling tools to complete tasks or updating different data stores independently.

Agents work on their own and don’t share results with each other. However, an agent can call other AI agents by running its own orchestration as part of its process. Agents need to know which other agents are available to work on tasks. This pattern allows you to either call all registered agents every time or choose which agents to run based on the specific task.

When to use concurrent orchestration

You may want to consider using the concurrent orchestration pattern in these situations:

• When tasks can run at the same time, either by using a fixed group of agents or by selecting AI agents dynamically based on what the task needs.
• When the task benefits from different specialized skills or approaches (for example, technical, business, or creative) that all work independently but contribute to solving the same problem.

This kind of teamwork is common in multi-agent decision-making methods such as:

• Brainstorming ideas
• Combining different reasoning methods (ensemble reasoning)
• Making decisions based on voting or consensus (quorum)
• Handling tasks where speed matters and running agents in parallel cuts down wait time

When to avoid concurrent orchestration

You may want to avoid using the concurrent orchestration pattern in the following scenarios:

• Agents need to build on each other’s work or depend on shared context in a specific order.
• The task requires a strict sequence of steps or predictable, repeatable results.
• Resource limits, like model usage quotas, make running agents in parallel inefficient or impossible.
• Agents can’t reliably coordinate changes to shared data or external systems while running at the same time.
• There’s no clear way to resolve conflicts or contradictions between results from different agents.
• Combining results is too complicated or ends up lowering the overall quality.

dynamics 365 finance training courses malaysia