Python With Power BI

Actually, when you need to create a bilateral data analysis you can use Power BI by Microsoft. It is not only interactive but also can visualize your information for your business intelligence. So that is why it is called BI or in other words Business Intelligence. It is useful because you can use Python with it.

Using Python you can enhance Power BI programing language capabilities. Python dashboard for data analysis, data acceptance, data conversion, data addition, and data visualization, also can use complicated functions such as machine learning libraries and more. This all can be done with numerous data dashboards with Python thankfully to these two mechanics Power BI and Python. Or in other words, thanks to Python BI Power.

Either experts or newbies can use this blog to learn Python BI Power. We will also use pandas and Matplotlib libraries.

Actually, pandas is an open-source library that can be used for operating with relative and termed data simply. It provides an interactive dashboard Python made from data and some functions for working with numbers and times. The core of pandas is the NumPy library and it is efficient and it has high performance.

And matplotlib is one of the excellent libraries that can help you visualize Power BI drag and drop dashboard for arrays. It is useful when you need to create a visualisation of big numbers. And also it has a lot of settings for your plots such as lines, bars, and even histograms.

So the answer to the question: When can you use BI Power? is “You can use them with a sales data dashboard using Python.”.

The Start in Using Power BI

As it is logical, Power BI code language is created by Microsoft so the only operating system where it runs on is Windows. But do not worry, you can use it with macOS or Linux distribution or any other operating system if it supports virtual machines. The version of Windows that supports Microsoft BI Power is Windows 8.1 and newer. Actually, if you are not using Windows as your main operating system, you need to have 30 gigabytes for a virtual operating system.

Installation of Power BI Desktop

Here we will try to set all tools and after that, we can make codes using Python. Microsoft Power BI Desktop is a powerful collection of tools and services that can be obtained, free of charge, without a Microsoft account, and with no need for an Internet connection. You can easily install it on your computer by accessing the Microsoft Store from the Start menu or its web-based storefront. With this amazing suite, you can work offline like a traditional office suite, giving you the convenience of having all the tools and services you need in one place. By installing Power BI Desktop from the Microsoft Store, you can ensure automatic and quick updates to the most recent versions of the tool without having to be logged in as the system’s administrator.

If the usual method of installing Power BI Desktop doesn’t work for you, you can always try downloading the installer from the Microsoft Download Center and running it manually. This executable file is about 400MB in size. Once you have the application installed, launch it and you’ll be met with a welcome screen. At first, the Power BI Desktop user interface may seem intimidating, but don’t worry. You’ll become accustomed to the basics as you progress through the tutorial.

What Python Code Editor to Choose?

To maximize your experience, why not install Microsoft Visual Studio Code? This free and modern code editor is immensely popular and can be easily found in the Microsoft Store. If you already use an IDE such as PyCharm or don’t require any of the advanced editing capabilities, feel free to skip this step. Otherwise, Visual Studio Code is a great choice for any coding enthusiast.

Microsoft Power BI Desktop offers only basic code editing capabilities, which is understandable given its primary purpose as a data analysis tool. Unfortunately, it lacks advanced features such as intelligent contextual suggestions, auto-completion, or syntax highlighting for Python, which are essential for writing anything but the most easy Python scripts in Power BI coding language. Consequently, it is highly recommended that you use an external code editor for writing more complex Python scripts Power BI.

You can download Visual Studio Code on any operating system without virtual machines and you can find it on the Microsoft website. The installation is simple as Visual Studio helps you when you download the installer.

VS Code is a cutting-edge code editor that brings seamless support for a wide array of programming languages via its extensions. Actually, it does not Power BI Python support from the box as PyCharm, but it will offer to install extensions to start working. Say goodbye to limitations, as VS Code doesn’t just stop at Python for Power BI. With its intuitive interface, it becomes a Python powerhouse once you open an existing Python file or create a new one, automatically recognizing the language and prompting you to install the best set of recommended extensions designed specifically for Python programming.

But all this works only if you have already installed raw Python on your computer. You can google it and find the answer to how to do this.

Does BI Desktop Need Some Libraries?

The answer is “Yes, it does”. Unleash the full potential of Power BI Desktop by ensuring that your Python setup is equipped with pandas and Matplotlib. These libraries are not part of the standard installation, but can easily be obtained if you’ve utilized Anaconda. It’s worth noting that incorporating third-party packages into the global Python interpreter is discouraged, as it can pose potential risks. Moreover, attempting to run the system interpreter from Power BI Python on a Windows machine is not possible due to permission restrictions. The solution? A Python virtual environment is a secure and efficient way to manage your Python packages and dependencies.

What is a Virtual Environment?

An isolated folder, a virtual environment, comprises a directory that comprises a replication of the main Python interpreter, allowing you to experiment with your heart’s content. You can install any additional libraries within this space without any concern of disrupting other programs that are reliant on Python. And, whenever you wish, you can effortlessly eliminate the folder holding your virtual environment without any adverse effect on the existence of Python on your machine.

To create a new virtual space you need to use a Windows terminal and write the command:

Here we named our folder “python-virtual” but you can use any you want.

In just a matter of moments, a fresh folder containing a duplicate of the Python interpreter will materialize on your desktop. With this, you can activate the virtual environment by executing its activation script, and subsequently install the two libraries that are required by Power BI. To achieve this, type in the following commands while your desktop remains in your present working directory:

Upon activation, you should be able to identify your virtual environment with the name “python-virtual” in the command prompt. Failure to do so would result in the installation of additional third-party packages into the primary Python interpreter, which is precisely what we aimed to avoid. Congratulations, you’re almost there! You can repeat the activation and pip installation steps if you wish to incorporate additional libraries into your virtual environment. Finally, the next step is to inform Power BI of the location of Python in your virtual environment.

Let’s Run Python on It

Firstly, we need to set special options in Power BI Desktop. Once you access the configuration options, you’ll find various settings organized by categories. Locate the category named “Python scripting” in the left-hand column, and proceed to set the Python home directory by selecting the “Browse” button.

To ensure proper functionality, it’s important to specify the path to the “Scripts” subfolder in your virtual environment, which contains the “python.exe” executable. If your virtual environment is located in your Desktop folder, your path should resemble the following format:

Before “\Desktop” must be the name of your user. If the designated path is incorrect and does not contain a virtual environment, you will receive an appropriate error message.

Great job! You have now successfully configured Python to Power BI. One key setting to verify is the path to your Power BI to Python virtual environment. It should include both the pandas and Matplotlib libraries. With this setup complete, you’re ready to start exploring the capabilities of Python with Power BI in the next section.

So next let’s talk about running the code and how it works.

How Can You Operate With It?

There are several methods available to execute Python and Power BI, each of which seamlessly integrates with a data analyst’s regular workflow. One such method involves using Python as a data science using Power BI source to import or create datasets within your report. Another method involves utilizing Python to perform data cleaning and other transformations on any dataset directly in Power BI with Python. Additionally, Python’s advanced plotting libraries can be used to create engaging and informative data visualizations. This article will explore all three of these applications in detail.

Using pandas.DataFrame

If you need to ingest data into Power BI from a proprietary or legacy system, you can use a Python  BI script tool to connect to the system and load the data into a pandas DataFrame. This is a useful approach when the data is stored in an obsolete or less commonly used file format that Power BI does not support natively.

To get started, you can write a Python script that connects to the legacy system and loads the data into a pandas DataFrame. Once the data is in a DataFrame, you can manipulate it using the pandas library to clean and transform the data as needed.

Power BI can then access the DataFrame by connecting to the Python script and retrieving the DataFrame. This allows you to leverage the power of both tools – the data manipulation capabilities of Python and the visualization and reporting capabilities of Power BI.

In this tutorial, we’ll use Python to load fake sales data from SQLite, which is a popular file-based database engine. While it is technically possible to load SQLite data directly into Power BI Desktop using an appropriate driver and connector, using Python can be more convenient since it supports SQLite out of the box.

Before jumping into the code, it would help to explore your dataset to get a feel for what you’ll be dealing with. It’s going to be a single table consisting of used car dealership data stored in the “sales.db” file.

Let’s imagine this file has a thousand records and a lot of columns of data in the table, which represent sold goods, their buyers, and the date of sold items. May you remember what we mentioned about Anaconda? Yes, Anaconda has a Jupyter Notebook that can be called a code editor. You can quickly visualize this sample database by loading it into a pandas DataFrame and sampling a few records in a Jupyter Notebook using the following Power BI Python tutorial:

Note that the path to the sales.db file may be different on your computer. If you can’t use Jupyter Notebook, then try installing a tool like SQLite Browser and loading the file into it.

At a glance, you can tell that the table needs some cleaning because of several problems with the underlying data. However, you’ll deal with most of them later, in the Power Query editor, during the data transformation phase. Right now, focus on loading the data into Power BI.

As long as you haven’t dismissed the welcome screen in Power BI yet, then you’ll be able to click the link labeled Get data with a cylinder icon on the left. Alternatively, you can click Get data from another source on the main view of your report, as none of the few shortcut icons include Python. Finally, if that doesn’t help, then use the menu at the top by selecting Home › Get data › More… as depicted below.

Doing so will reveal a pop-up window with a selection of Power BI connectors for several data sources, including a Python script, which you can find by typing Python into the search box.

Select it and click the Connect button at the bottom to confirm. Afterward, you’ll see a blank editor window for your Python script, where you can type a brief code snippet to load records into a pandas DataFrame.

You can notice the lack of syntax highlighting or intelligent code suggestions in the editor built into Power BI. As you learned earlier, it’s much better to use an external code editor, such as VS Code, to test that everything works as expected and only then paste your Python code to Power BI.

Before moving forward, you can double-check if Power BI uses the right virtual environment, with pandas and Matplotlib installed, by reading the text just below the editor.

While there’s only one table in the attached SQLite database, it’s currently kept in a denormalized form, making the associated data redundant and susceptible to all kinds of anomalies. Extracting separate entities, such as cars, sales, and customers, into individual DataFrames would be a good first step in the right direction to rectify the situation.

Fortunately, your Python script may produce as many DataFrames as you like, and Power BI will let you choose which ones to include in the final report. Here in Power BI programming language code, you can extract those three entities with pandas using column subsetting in the following way:

First, you connect to the SQLite database by specifying a suitable path for the car_sales.db file, which may look different on your computer. Next, you run a SQL query that selects all the rows in the sales table and puts them into a new pandas DataFrame called df. Finally, you create three additional DataFrames by cherry-picking specific columns. It’s customary to abbreviate pandas as pd in Power BI coding. Often, you’ll also see the variable name df used for general, short-lived DataFrames. However, as a general rule, please choose meaningful and descriptive names for your variables to make the code more readable.

When you click OK and wait for a few seconds, Power BI will present you with a visual representation of the four DataFrames produced by your Python script.

The resulting table names correspond to your Python variables. When you click on one, you’ll see a quick preview of the contained data. The screenshot above shows the customers table, which comprises only two columns.

Select cars, customers, and sales in the hierarchical tree on the left while leaving off df, as you won’t need that one. You could finish the data import now by loading the selected DataFrames into your report. However, you’ll want to click a button labeled Transform Data to perform data cleaning using pandas in Power BI.

In the next section, you’ll learn how to use Python to clean, transform, and augment the data that you’ve been working within Power BI.

Using Python Query Editor

If you have followed the instructions in this guide, you should now be in the Power Query Editor, which displays the three DataFrames you selected earlier. These DataFrames are referred to as queries in this particular view. However, if you have already imported data into your Power BI report without applying any transformations, there’s no need to worry! You can access the same editor at any time.

To do so, navigate to the Data perspective by clicking on the table icon located in the center of the ribbon on the left-hand side, and then select Transform data from the Home menu. Alternatively, you can right-click on one of the fields in the Data view on the far right of the window and choose the Edit query for the same result. Once you have accessed the Power Query Editor window again, you will be able to see your DataFrames or Queries on the left-hand side, while the Applied Steps for the currently selected DataFrame will be displayed on the right-hand side, with rows and columns in the center.

Each step in the Applied Steps represents a sequence of data transformations that are applied in a pipeline-like fashion against a query, from top to bottom. Each step is expressed as a Power Query M formula. The first step, named Source, involves invoking your Python script, which generates four DataFrames based on the SQLite database. The other two steps extract the relevant DataFrame and transform the column types.

By clicking the gear icon next to the Source step, you’ll reveal your data ingestion script’s original Python source code. This feature can access and edit Python code baked into a Power BI report even after saving it as a .pbix file.

You can insert custom steps into the pipeline for more granular control over data transformations. Power BI Desktop offers plenty of built-in transformations that you’ll find in the top menu of Power Query Editor. But in this tutorial, you’ll explore the Run Python script transformation, which is the second mode of running Python code in Power BI:

Conceptually, it works almost identically to data ingestion, but there are a few differences. First of all, you may use this transformation with any data source that Power BI supports natively, so it could be the only use of Python in your report. Secondly, you get an implicit global variable called dataset in your script, which holds the current state of the data in the pipeline, represented as a pandas DataFrame.

Note: As before, your script can produce multiple DataFrames, but you’ll only be able to select one for further processing in the transformation pipeline. You can also decide to modify your dataset in place without creating any new DataFrames.

Pandas lets you extract values from an existing column into new columns using regular expressions. For example, some customers in your table have an email address enclosed in angle brackets (<>) next to their name, which should belong to a separate column.

Select the customer’s query, then select the last Changed Type step, and add a Run Python script transformation to the applied steps. When the pop-up window appears, type the following Python script code example:

When working with Power BI, you can utilize the implicit dataset variable in your script to reference the customer’s DataFrame, giving you access to its methods and allowing you to override it with your transformed data. Alternatively, you have the option to define a new variable for the resulting DataFrame. During the transformation process, you can add two new columns, full_name and email, and then remove the original customer column containing both information pieces.

Once you’ve finished your transformation, clicking OK and waiting a few seconds will display a table showing the DataFrames your script produced. In this case, there is only one DataFrame named dataset, as you reused the implicit global variable provided by Power BI for your new DataFrame. To choose your desired DataFrame, simply click the yellow Table link in the Value column.

Your customers` table now has two new columns, allowing you to quickly identify customers who have not provided their email addresses. If you desire further transformations, you can add additional steps. For example, you could split the full_name column into separate columns for first_name and last_name, assuming that there are no instances of customers with more than two names.

Be sure to select the final transformation step and insert another Run Python script. The corresponding Python code for this step should appear as follows:

Unlike in the previous step, the dataset variable refers to a DataFrame with three columns, full_name, and email, because you’re further down the pipeline. Also, notice the inplace=True parameter, which drops the full_name column from the existing DataFrame rather than returning a new object.

You’ll notice that Power BI gives generic names to the applied steps and appends consecutive numbers to them in case of many instances of the same step. Fortunately, you can give the steps more descriptive names by right-clicking on a step and choosing Rename from the context menu:

By editing Properties…, you may also describe in a few sentences what the given step is trying to accomplish.

When you’re finished transforming your datasets, you can close the Power Query Editor by choosing Close & Apply from the Home ribbon or its alias in the File menu:

This will apply all transformation steps across your datasets and return to the main window of Power BI Desktop.

Next up, you’ll learn how to use Python to produce custom data visualizations.

Power BI Python Data Transformation

So far, we’ve covered importing and transforming data using Python in Power BI Desktop. Python’s third and final application is creating visual representations of your data. When it comes to visualizations, you have the flexibility to use any of the supported Python libraries, provided you’ve installed them in the virtual environment that Power BI utilizes. However, Matplotlib serves as the foundation for plotting, which other libraries delegate to in any case.

If Power BI hasn’t already directed you to the Report perspective following your data transformations, you can now navigate by clicking on the chart icon on the left ribbon. This will bring up a blank report canvas where you can add your graphs and other interactive components, collectively referred to as visuals.

Over on the right in the Visualizations palette, you’ll see several icons corresponding to the available visuals. Find the icon of the Python visual and click it to add the visual to the report canvas. The first time you add a Python or R visual to a Power BI report, it’ll ask you to enable script visuals.

In fact, it’ll keep asking you the same question in each Power BI session because there’s no global setting for this. When you open a file with your saved report that uses script visuals, you’ll have the option to review the embedded Python code before enabling it. Why? The short answer is that Power BI cares for your privacy, as any script could leak or damage your data if it’s from an untrusted source.

However, if you’ve configured Power BI to use an external code editor, then clicking on the little skewed arrow icon (↗) will launch it and open the entire scaffolding of the script. You can ignore its content for the moment, as you’ll explore it in an upcoming section. Unfortunately, you have to manually copy and paste the script’s part between the auto-generated # Prolog and # Epilog comments back to Power BI when you’re done editing.

Note: Don’t ignore the yellow warning bar in the Python script editor, which reminds you that rows with duplicate values will be removed. If you only dragged the color column, then you’d end up with just a handful of records corresponding to the few unique colors. However, adding the vin column prevents this by letting colors repeat throughout the table, which can be useful when performing aggregations.

To demonstrate an elementary use of a Python visual in Power BI, you can plot a bar chart showing the number of goods painted in a given color. Here is an example of import visualize Python:

To get started with creating visualizations, you can begin by enabling Matplotlib’s theme that mimics the seaborn library. This will provide a more visually appealing look and feel compared to the default theme.

Next, you can remove any records with missing color data, and count the number of remaining records in each unique color group. This will result in pandas.Series object that can be plotted and color-coded using its index, which consists of the color names. Finally, you can render the plot by calling plt.show().

With these steps, you can easily create a basic visualization of your data using Python in Power BI. Of course, the possibilities for visualizing your data are endless, and you can explore and experiment with other Python libraries and techniques to create even more engaging and informative visualizations.

Additional Settings For Power BI Desktop

With the power of pandas and Python, there are countless possibilities for transforming your datasets in Power BI. Some examples include:

• Anonymizing sensitive personal information, such as credit card numbers
• Identifying and extracting new entities from your data
• Rejecting sales with missing transaction details
• Removing duplicate sales records
• Unifying inconsistent purchase and sale date formats

These are just a few ideas to get you started, but the possibilities are endless. While we can’t cover everything in this article, don’t hesitate to experiment on your own. Keep in mind that your success in using Python to transform data in Power BI will depend on your understanding of pandas, which is the library that Power BI uses under the hood. The more you learn about pandas and their capabilities, the more you can achieve with your data in Power BI.

Special Code Editor

Within the Python scripting options in Power BI, a useful setting allows you to specify the default Python integrated development environment (IDE) or code editor you prefer to use when working on a code snippet. You can stick with the operating system’s default program associated with the .py file extension, or you can select a specific Python IDE of your choice to launch within Power BI. This flexibility can make it easier and more efficient for you to write and debug Python code directly in Power BI.

To indicate your preferred Python Integrated Development Environment (IDE), opt for “Other” from the initial dropdown menu, and navigate to the executable file of your preferred code editor. For instance, you may browse this one:

As before, the path to your app can be different and contain different folders.

What are the Cons of Using Python Power BI?

Python integration in Power BI Desktop has some limitations you should be aware of.

Timeouts

The most notable limitations are related to timeouts, data size, and non-interactive visuals. Your data ingestion and transformation scripts defined in Power Query Editor can’t run longer than thirty minutes. Python scripts in Power BI visuals are limited to only five minutes of execution, and there are additional data size limitations, such as only being able to plot the top 150,000 rows or fewer in a dataset and the input dataset can’t be larger than 250 megabytes.

Marshaling

In Power BI Desktop, the communication between Power BI and Python is done by exchanging CSV files. Therefore, when using Python to manipulate data, the script must load the dataset from a text file created by Power BI for each run, and then save the results to another text or image file for Power BI to read. This redundant data marshaling can result in a significant performance bottleneck when working with larger datasets. It is the biggest drawback of Python integration in Power BI Desktop.

If you encounter poor performance, you may want to consider using Power BI’s built-in transformations or the Data Analysis script Expression (DAX) formula language instead of Python. Another approach to improve performance is to reduce the number of data serializations by collapsing multiple steps into a single Python script that does the heavy lifting in bulk. For example, instead of making multiple steps in the Power Query Editor for a very large dataset, you can combine them into the first loading script.

Python Visualization

Data visualizations created using Python code are static images, which means you can’t interact with them to filter your dataset. However, Power BI will update the Python visuals in response to interacting with other visuals. It’s worth noting that Python visuals take slightly longer to display due to the data marshaling overhead and the need to run Python code to render them.

And Others

Using Python in Power BI has some other minor limitations. For instance, it can be difficult to share Power BI reports that rely on Python code with others, since the recipients would need to install and configure Python. Additionally, all datasets in a report must be set to a public privacy level for Python scripts to work properly in the Power BI service. Furthermore, a finite number of supported Python libraries in Power BI exist. There may be additional minor limitations, which can be found in Microsoft’s documentation that outlines how to prepare a Python script and known limitations of Python visuals in Power BI.

What are Anonymous Types in C#

Anonymous types are a mighty instrument in object-oriented programming disciplines. In strongly typed programming languages like C#, we need to always define the type of a variable before we can create new ones. Sometimes we have to create a new instance of an object with an unknown type and read-only attributes, which can be done only with explicitly undefined types.
By clearly describing small things, they can encapsulate attributes into an object without defining a type. And every single object can hold any combination of attributes, regardless of their data type, as long as each attribute is clearly defined and described.
They are perfect for the execution of SQL-like LINQ because they return an unspecified type to define the new object on the fly. It can help you streamline your C# and .NET code while also providing developers with a powerful tool for creating unidentified anonymous objects with a predefined structure.

How to Use Anonymous Types

Anonymous typing simplifies development and produces more readable and clean code. In combination with generics, you can create strongly typed collections, which can be useful with LINQ queries, passing attributes, or creating models in ASP.NET MVC.
Anonymous typing doesn’t work with inheritance or interface implementation. If you need a more complex type that requires functions or fields, your best choice is to define a class.

C# Anonymous Types Example

We made an array called “hackers” that has two objects in it with the values “Type” and “Exp“, a list called uniqueHackers, and a hash set called seenTypes.

The code iterates through the hackers array, and for each unique type, finds the maximum experience among all hackers. Finally, it creates a new object with a unique type and maximum experience and adds it to the uniqueHackers list.

This code will find the hacker(s) with the highest experience for each type of hacker. Note that it is a list of anonymous types, so you won’t be able to access the properties directly by name. Instead, you can use reflection or dynamic typing to access the attributes.

Nested Anonymous Type in C#

Anonymous types can’t be used outside of the scope where they are defined, so they may not be suitable for more complex scenarios or for sharing data across multiple methods or classes.

This code is more concise and easier to read, especially when dealing with simple data structures. Here’s a code example of how you could create a nested anonymous type from the people array in the original code. It’s important to note that code readability can suffer if there are too many nested levels. In such cases, it may be preferable to utilize named types or nested classes instead.

In this code example, the people array contains two objects, each with Type, Exp, and Address, which itself is an anonymous object with Street, City, State, and ZipCode properties. When dealing with complex data structures, we can utilize dot notation to access the nested properties like this:

Anonymous Types in LINQ

LINQ can significantly simplify the code and enhance its readability when you work with complex data structures. To create an anonymous array using a LINQ query, modify the code as follows:

This snippet creates a new anonymousArray with Type and Exp properties. This code uses LINQ to sort by maximum experience, then creates an array with unique hackers, similar to the original code. There we have concise and readable code while still achieving the desired functionality.

Note that the order of the elements in uniqueHackers is not guaranteed, so we set the properties of each element in anonymousArray explicitly to ensure they match the original hackers array.

Why Infrastructure As A Code Tools Used In Cloud Platforms?

IaC is a kind of methods used to control and describe centers of data processing with data sets for configuration rather than using manual methods of editing configurations on servers or interrelationships of infrastructures. Normally, the declarative method or imperative algorithm is used in writing code for operating with infrastructures.

Infrastructure as code (IaC) is a common set of tools used in cloud computing. The core principle of the IaC is to describe an infrastructure with a code in combination with an ordinary software development procedure. It is the main practice of developers and a component of a continuous software supply. IaC allows DevOps to work as a team promptly, reliably, and on large scales. They can use a single set of methods and tools to develop programs and servicing infrastructure.

From Hardware to Cloudformation tools

Many people probably no longer remember the iron age when we had to buy our own servers and computers. At that time no one had any idea what tools for cloud formation were. Now it already seems crazy when the hardware buying cycle can limit the infrastructure growth. A new server used to be delivered and installed for weeks! The software was available to developers many days after the hardware was installed.

The first cloud computing methods appeared only in the middle of the 2000s. This made it possible to run new instances of virtual machines quickly and brought businesses and developers not only benefits but also problems. First and foremost they had to maintain an increasing number of servers. However, these were still far from IaC tools.

Following that, only some large computers began to be replaced by smaller ones and the area of the infrastructure of an average engineering center began growing and became more cyclical. Ops had to support more and more things. In order to cope with the peak load, it was necessary to make up or down scaling at different times of the day.

To increase efficiency, many pickets had to be created in the morning to achieve the maximum power and many also at night to reduce that power. The whole process had to be managed manually, which became a challenge over time.

All abovementioned was the reason for the creation and introduction of the infrastructure as code tools. This allowed the systematization of the listed above task maximally. IaC solution made the management of the data processing centers and servers very sufficiently with the help of data readable by computers. They became an alternative to physical equipment and tool configuration under human supervision.

Amazon Web Cloud Formation Service (AWS) was the first tool that emerged in 2009. It became one of the best tools for DevOps allowing engineers to create versions of infrastructures as quickly as a normal code can make it as well. And that allows tracing the resulting versions of infrastructures in order, to make environments enough consistent.

Well-known Infrastructure As Code Tools

The TOP IaC tools that become famous recently among developers are:

• Terraform IaC
• Amazon Web Service Cloudformation tools (AWS)
• Azure Resource Manager
• Ansible
• Chef
• Puppet
• Pulumi
• Saltstack
• Google Cloud Deployment Manager
• Vagrant
• Crossplane

Now, we would like to compare all aforementioned infrastructure as code tools to understand what similarities or differences they have concerning the application area, writing method, or languages.

Main Infrastructure As Code Tools Tasks

At the present time, it is hard to imagine the work of major providers and services without a cloud automation tool application. A wide range of IaCs is dedicated to helping IT engineers to solve such challenges as:

• Deployment
• Instrumentation
• Configuration
• Provisioning

Earlier IT specialists set, configured, and updated software for cloud servers manually. Team participants stored and configured data also with the same method. It took much time and required the attraction of additional developers and influenced significantly the increase in expenses.

IaC infrastructure as code became a solution for professionals in addressing such problems as additional expenses for salary payment and solving problems with the scalability.

It is worth being aware that some IaC tools are already set inside the settings of the infrastructure and other kinds of tools manage applications and infrastructure in the environment.

Below, we would like to give a few words about AWS infrastructure as code and its advantages.

AWS Infrastructure As Code Advantages

The IaC is aimed at the provision and management of cloud resources using a template read by people, which can be consumed by machines in an easy way. AWS Cloud formation is considered a reliable solution to the DevOps cloud services, which uses the IaC for Amazon Web Services.

AWS type of cloud formation enables creating a personal account of a user on Amazon Web Services using the description requested by a user. Then, this description is realized upon request. A typical infrastructure as code example includes a fragment of the template, which describes the creation of resources for the Amazon Elastic Computed Web Services using YAML.

Therefore, when we create the code, we indicate the AWS, then ECS and Service gradually as a Type, then put the “Discovery of the service” as the Dependence, also indicate in the properties the “App” as a name, “Production” as the cluster, 200 maximal percent and 75 minimal percent in the deployment configuration and set 5 as the number of counts.

AWS cloud formation tools then take the template and after that becomes responsible for the creation, updating, and removing resources on a user’s Amazon Web Service account depending on the content of this template. If a user wants to add a new resource to his file, the Cloudautomation tool builds this resource in his account. In case this user wishes to update his recourse, the tool can update or replace all current existing resources. If a user wants to delete this resource from his template, it will be vanished or be removed from his account.

Tools IaC provides users many pros:

They are visible:

An IaC template plays a big role as a precise reference on what kinds of resources you have on your account and their indicators. To check settings there is no need to follow the web panel.

They are scalable:

You can write the infrastructure as a code one time and then use it multiple times. It means that you can use just a good quality template as the basis for different services in various areas of the world, which significantly simplifies horizontal scaling.

They are stable:

If a wrong parameter or a wrong resource has been removed from the web panel, you can break everything. IaC tools solve this problem, especially in combination with Git versions for control.

They have transaction ability:

Cloudformation tools can not only create resources in your AWS account, but they also wait until their stabilization during the starting process. IaCs check for a successful initialization and in case of any failure, they can roll carefully the infrastructure back to the previously known good condition.

They are secure:

It can be seen that the provides of IaC again you with a single template to deploy your architecture. As soon as your protected architecture has been created, you can use it many times and you will know that each deployed version can have the same settings.

Conclusions

IaCs are very popular instruments of the new generation introduced at the beginning of the new century to make the process of cloud service formation, deployment, and adjusting of the infrastructure easier using just a code. There is no need to make manual settings, which significantly simplified the tasks of developers and solve a problem with scalability as well. Terraform and its closest analog Pulumi is considered the most common tools used for cloud formation.

Binary Search Tree in Python (BST)

In this review, you will grasp more information about Binary search trees (BST). There will also be instances of BST in Python. Before starting to learn it, you need to familiarize yourself with what a Binary search tree is.

Python binary search tree uses a data structure that lets us keep values in a sorted list. Here each node is variable and this variable is bigger than its left child nodes. It is called binary sort in python.

There exists two reasons why this tree structure Python is called a Binary search tree. Firstly, each tree node has a max amount of child nodes of two (so that is why it is called a binary tree). Secondly, in O(log(n)) time search binary tree Python can be used to find a number (so that is why it is called a search tree).

The qualities that Binary search tree in Python has, and which make them different from regular binary trees:

• Nodes which count as lower than the root node can be found in a node’s left subtree.
• Nodes which count as bigger than the root node can be found in the right subtree of a node.
• The both subtrees must be Binary search trees and possess the aforementioned characteristics.
• There cannot be any duplicate nodes in binary search trees.

A tree has a right subtree with one value smaller than the root is shown to demonstrate that it is not a valid binary search tree program in Python

The binary tree above isn’t a valid  Python binary search tree because the node “10” right subtree contains a value “9” that is smaller than it. The second property of BST Python is violated. But if we change the value “9” to “11”, it will be a Binary Python search tree. So let’s explain binary search tree with example:

Now, what can we do with a binary search python program? So actually binary tree search Python can be used for storing something like data. The plus will be that it will be organized so you can without a doubt insert something, delete, and update. And also these operations will be as fast as lightning. As we said before BST can provide binary tree search complexity that is called Big-O of O(log(n)) when you are searching, updating or deleting data. Actually linear O(n) is slower than log(n) and it needs some time to find elements. And fact says that a lot of productions use binary trees in their databases like MySQL or PostgreSQL to speed them up when they use CRUD operations.

There are basic operations that you can perform on a binary search tree. Here is the binary search tree implementation. Below we will show basic operations that you can use with a binary search tree.

Let’s create a binary tree python. Tree node class python code can be used as a pointer so you can point to a root node that will connect to the child nodes. Here is the binary tree program in python:

Here we will have a value that can function as a key. Also in the line of tree in python code:’value=None’ we can see that if a value will not be created, it will count as None. And in the next two lines, we create two child nodes that equal ‘None’.

Linear search in Python Binary search tree

Binary search algorithm in python depends on the quality of the Binary search treе that all of the left subtrees have values below the root node and all of the right subtrees have values above the root node.

Searching in Python for a value in a binary tree involves comparing the incoming value with the nodes. If this value is below the root node, it means that the value is not in the right subtree, so the search will take place in the left subtree accordingly if this value is above the root node, it means that the search will take place in the right subtree.

So now, let’s try to show steps on how it works. We are finding the number ‘11’:

Here our search method in Python has not found the number eleven, so it transfers to the right subtree to find it:

Our search program in python has not found the number eleven here, so it transfers to the right node again because ‘10’ is a higher number than eleven:

The search program in python has not found again the number ‘11’, and the number ‘12’ is higher than ‘11’ so it transfers to the left subtree:

Finally, the number ‘11’ was found. Well done, binary tree search!

When the binary tree in Python finds our value, it returns. The returned value will spread in every step, so that means that if there will not be, for example, ‘12’, it will find it anyway. In situations when a binary tree can not find the value you need, it returns NULL. Here are the binary search algorithm steps:

Let’s write a program for binary search in a data structure.

So here we see the easy search code in python that returns the boolean value True or False. Binary search python code tree depends on whether the value that we want to find exists or not.

Insert operation in Python Binary search tree

When we want to insert something, for example, a value, we use the same technique as searching because we obey the only rule that in the right subtree there must be larger values and in the left subtree there must be lower values than in our root node.

So the insert technique differs from searching in putting the new node. It looks like that we go through each subtree depending on the value and we put the new node when the left or right subtree is NULL.

Example of binary tree code in python:

So now, let’s try to show steps on how it works. We are going to add the number ‘11’:

Here we can see that ‘11’ is higher than our root ‘7’ so the Python binary search program transfers to the right subtree:

Now again ‘10’ is higher than ‘11’ so we transfer to the right:

Here is the last transfer, because the number ‘11’ is lower than ‘12’ so which means we’re gonna transfer to the left subtree. And we see that the left subtree is NULL so we add ‘11’ here. Hooray!

So we put the node with the number ‘11’, but we did not exit the operation, because we needed to return the value. So here the return node comes in hand. The process of returning a node works when we have NULL and create the new node that is returned and attached to the parent node. If we have not found the position for a new node, we without change come back to the root in reverse order.

That means that we move in reverse order in the tree without changing other connections.

Here we see how in each step we come back to the root without changing any value.

Also, let`s try to create a Python binary search code for this operation:

Here we see that we created a python program for binary search where the given value sets and returns if the node does not own a value. But if our node contains the value, it returns it. Also, we used the rule that if the value is lower than the value that is in our node and we have a left child, we recursively call insert on this child. In the opposite case, if we do not have the left child, we just create it with the value that was given. And all the same, is for the right child, but only if our value is higher.

Delete operation in Python Binary search tree

The operation of deleting the node has three variations:

First BST variation:

Binary search in Python program deletes the node that was located in the end of the tree and does not change any connections. For example, we want to delete the number ‘11’:

Here python binary search algorithm detected the value and deleted it:

Second BST variation:

When we delete the node that has a child node. For example, we need to delete the number ‘12’:

Here we detected the number ‘12’. Next, we copy the value of its child. Our child is number ‘14’. So we copy this value:

We have already copied the child ‘14’.Then we do the same step as in ‘First variation’, we delete the last node:

Third BST variation:

When we have more than one child below the node we are searching for deleting. For instance, we need to delete the number ‘10’ from the tree:

Here we detected the value. Now we must find the successor of the number ‘10’ in the tree. It is ‘11’, so we copy the value:

Then we finally delete the node of successor ‘11’:

Let’s try to make the program for binary search in python with delete operation:

So here we see this huge tree in Python data structure. But do not worry, it is simple. Actually, this operation works recursively, but it returns the new shape of the node that was given after doing the delete. And it helps in giving access to the parent node whose child node was deleted so we can properly set where the data must be in the left node and where it must be in the right node.

Getting minimum and maximum

with Python Binary search tree

Here we see a very straightforward tree class in Python that can be helpful. They transfer the edges of the data structure tree Python so we can find the largest and the smallest values in our storage.

Why would you need binary trees?

Let’s define a python tree node class. Binary search trees Python can provide very fast O(log(n)) operations that were mentioned before. They are useful for binary sorting in Python. It is very simple to understand and very useful. A simple node class in python without any operations needs only a few lines of code to get to work.

What about the cons? Binary search trees in data structure python are slow for force-brute search. If you want to have iterations through each node, you must better use arrays. Also, BST requires more memory than arrays in implementation.

Actually, there are a lot of appliances where the data tree python will be useful. So the easiest solution to the problem of storing indexes and keys in the database will be BST.

Let’s imagine the example when you need to create a main key column in SQL databases. So here you can without a doubt use a binary python class tree where keys are values in the column and you can point to the rows with nodes. In this situation your SQL base will be easy to search by using keys.

For example, you need to create a game where there must be a list of records of users. It is easy to find the nickname of the user by using binary tree search. And there are more apps where class tree python will be useful. Other simple uses for binary trees you can find in Google.

But what should I use? Binary search in python using list or the binary trees? For this question there are a lot of different answers. They are both positive and negative. Firstly, python node tree use the same pointers to track where the node is as linked lists. But node class python is more efficient and faster while you are searching.

What is Getter and Setter in Python?

After using C++ or other languages you can know how to define getters and setters. But if you start learning the first language Python, you can see that they are not so popular. Often people use them for attributes for classes. They help in creating access for private attributes while preserving encapsulation. But in Python you usually use them to uncover attributes partially in public API and use things when you need attributes with useful actions.

Although it is in Pythonic style to use properties, they can anyway have disadvantages. That’s why people start using getters and setters Python more than properties. And also you must know some OOP or in long: object-oriented programming.

After specifying the class in object-oriented programming, usually you use class attributes and instances. And the attributes that you have created are variables that you can use using instances or the class or both.

Firstly, let’s try to create a class in Python.

How to Create Class

If you need to create the class first you need to use the class name and variables. Here is an example:

Here we have created the class with the name Example. Let’s create objects from this class:

So here the object was created which is called Example1 and it is from the Example class.

That all means that when we printed Example1.first, we had got the value that was stored in a variable in the class that we had created. This calling and these variables are our attributes of the class.

Python Class Attributes

So the basic getter definition of attributes in class is that variables in class can be inherited by objects in the class. The next examples will show how it looks in code, but the only thing we must say is that those attributes are written outside the function __init__().

Let’s see the example:

So here we created the variable called ”office” in the class called Example. Then we used ”init” to create two more variables that are “name” and “position”. “Self” parameter inside init helped us to initialize them.

Python Setter Getter Strategies

So what are a getter and setter? Subsequently, the nation of the objects is held in attributes. In case you want to get admission to this kingdom, you need to have an entry to these attributes. So to get entry to those attributes you can use techniques or access them without delay.

After revealing them to different users, these attributes become part of the elegance’s API. So public attributes imply that other customers can have entry to the attributes in their snippets of codes.

Whilst you are in this example, languages like C++ want to use some techniques to address attributes in lessons. And as you can determine, they may be setters and getters.

Strategies of getter and setter Python have become properly appreciated in programming languages. Human beings use them for item-oriented programming, so surely there are a lot of human beings who’ve already heard about them.

So what are getters setters? If we need to talk about the usage of easy definitions, we are able to say that getter is the method that allows you to have an opportunity to have access to the values in a characteristic. And setter is the method that helps you to change or insert the cost inside the attribute.

Our everyday techniques submit that attrs which are public are operated when you have decided that nobody from builders will ever contact them. While you need to change the inner implementation of the attribute, you will use setter getter Python methods. To enforce the setter and getter it’s required to make the attributes non-public and write each characteristic with setter and getter. For example, say that you want to jot down a Room magnificence with width, length, and height attributes. Permit’s use getters and setters:

For instance, say that you need to jot down a python class getter setter with width, length, and height attributes. Let’s use getters and setters:

Here we can see that the constructor of our class uses three arguments that are width, height, and length. And we saved them in “._width”, “._length”, “._height” non-public attributes.

Often, you can use getter Python methods for returning the focused value of an attribute. And setter methods you use to make a new value and change it in this attribute.

Also, when you are used to programming C++ you can see that Python does not have private, protected, or public modifiers. Python has only public or non-public.

When you want to mark the attribute as non-public, you can use Python in-built syntax that is written before the name. It looks like the underscore “_”. This is only syntax, so it does not ruin all your plans of using dot notation like “object._attribute”. Dot notation is used often in “self.name” or anything else.

Also to use your code and make it work you can use “import”:

Let’s explain what we have done here. As we know the Pythonic way to add behavior for attributes is to make them a property. Properties percent together strategies for getting, setting, deleting, and documenting the underlying data. In short, properties we can call as attributes that have additional behavior.

Actually, it’s not a problem to use properties in the same way as stock attributes. Whilst you get entry to a property, its attached getter method is robotically called. And whilst you mutate the property, its setter technique is called.

The room gives us the opportunity to make instances so we can make changes to the associated height, width, and length.

Features of Setter and Getter

Using a setter and getter only for accessing or changing the values is not the only way. When you want to save all values in uppercase you can also use them. Well, you may use it by attribute ”.text” but you can’t. So if you want to use it, it seems that you have come from Java and C++ because they do not support constructs that are like properties. So that’s the issue why you can not use your attributes here as public. You should use setter and getter and provide implementation without public.

Python properties have plenty of capacity use cases. For instance, we can make properties to form read-write, read-only, and write-only attributes. Properties will let you delete and record the underlying attributes and also more interesting is that you can make them act like controlled attributes with attached behavior without converting the way you figure with them.

Because of Python possibilities when you use properties, people generally tend to use them. So if you need to add some new processing with attributes you can use them. But when you want to change all your attributes, you will waste a lot of time. And even it can overload your code and make it less efficient.

Descriptors in Python

Now after we get to know the base of topics about getter setter Python, also about properties that are written in a Pythonic way and also dealing with problems with how to add more functionality, we can learn other tools and techniques that we may use when we want to change getter and setter in Python on other methods.

So a descriptor is a feature in Python that helps you in creating attributes with behaviors. If you want to create a descriptor, you must use the specified protocol. In other words, it is a syntax that helps you. They are “.__get__()” and “.__set__()” methods.

Factual descriptors are very similar to properties in Python. Or in other words, Python class property setter is a type of descriptor that is more powerful than an in-built property. To clarify a case of how to use descriptors to create attributes with useful behavior, let’s use the example before:

Here we have used “@property” and “@.setter” which allowed us to change the height, width, and length for each room. In general, in case you locate yourself disordering your instructions with comparable property definitions, you ought to not forget to use a descriptor instead.

Methods __setattr__() and __getattr__()

Except traditional getters Python and setters you can use .__setattr__() and .__getattr__() special methods that help you to manage the attributes. Let’s make a class named Position that will change the type of x and y into float:

Here we can see that __init__ takes our position coordinates “.__getattr__()” method returns the firstname that is the coordinate. To make this done we used “.__dict__”. Take note that Python use  “.__getattr__()” automatically when you try to access an attribute with dot notation. What about the “.__setattr__()” method is that adds or changes the attributes. For example we used it for operating with each positional coordinate and converting it into float type. And also as with “.__getattr__()”, “.__setattr__()” runs automatically when you use assignment operation.

Here in our class we have to get or set accessors expected with our positions as regular attributes.

So here we used a very interesting example and showed how it works. But actually, you will not use similar. The tools that we have used help you to make validations and transformations with access to our attribute.

And in short, “.__getattr__()’’ and “.__setattr__()” are like an implementation that is like a Python getter and setter pattern. But actually, they work the same as regular methods getters and setters.

What Does Inheritance Look Like?

If you want to inherit some classes in Python you will meet up with one problem. For instance, you want to change the Python property method in a subclass. And if you try to Google it, you do not find any safe ways to do this. In short, it is hard to overturn the regular getter and wait that this property getter and setter will have the same functions as in the parent class. So that means that when you try to change the method in the inherited class you change the whole property including its other methods as a setter. Let’s see the example:

Here we can see that in class “Room” we changed the @property. This case shows us that we changed the whole .title property, including example setter too. Let’s add some functions:

Here we can see how Python can not find the setter of the inherited class. This is because we have changed it before and we are now trying to use a false setter, but he was not inherited. So, what is the solution to this situation?

Let’s use regular get set in Python methods:

Here we see that no issues were found. Python has an in-built independent get and set in Python methods. Room is a subclass of Goods and it overrides the getter. We can see that this line of code does not change the functionality of setter and successfully it was inherited from the parent class.

Let’s make an example with get set Python and functions:

Now our class can work properly. The changed getter method in the inherited class is functional. And the setter method also works well. Well done!

What is Better? Set and Get in Python or Properties?

If you start coding, you know the answer. There are some situations where methods of setter and getter are preferred over properties, but also there are other situations where Pythonic properties play better.

Let’s try to distribute by points why the get and set in Python are dealing with better, we can do this. When we talk about transformations, the methods setter and getter run better on attribute Python accessors and mutation Python. Also, they use inheritance and this is very good. Getter and setter can raise Python exception attributes when they are accessing the attributes. And finally, their integrations are being facilitated in heterogeneous enhancement teams.

When we want to talk about properties in Python, the problem can be seen when you are making a huge amount of lines of code. In other words, take note that you should avoid making slow Python mutations with Python properties. When somebody repeatedly tries to make access to a given attribute, users can suffer from waiting because it can take a long time, and the efficiency of your code can suffer a lot. So when your plans are to use Pythonic properties to change values in attributes, be sure that your mutations are fast, because if you will not do that and use slow get set accessors, you will change your code to use a simple setter and getter.

Another situation is that getter and setter methods are more flexible than you would think. Let’s make an example with the name Goods and its expiration date attribute:

Our date was a constant value and we made it read-only, so it means non-public. Also because of the fact that human error exists, so in life, you will face a lot of cases where somebody writes letters instead of numbers in the date section. Here we provided regular getter and setter methods for the “.expiration_date” attribute. Here our setter method has taken the “force” argument so that we can be ready for mistakes made by people. Also factually, traditional setters do not use more than one argument in code, so we can say that our code is wrong.

Here we have written the code without the “force” argument and got a message. Let’s make it work properly:

In the first example we tried to change the expiration date by using .set_expiration_date. But the problem was that we did not use force argument – we have not changed its setting to True. That is why we have got an AttributeError. So in the next example we replaced its setting and said to Python that we are sure that there are no mistakes. So it worked.

As we said before, the Python property setter getter does not accept more than one argument in the setter, because it accepts only that variable that you want to change or make.

Usually, you won’t accept a work message like “objects.attributes = value” to heighten an exception. Especially, you may accept messages that heighten exceptions in response to mistakes. Here regular Python getter setter methods were extra specific than properties.

Let`s look like on line: “phone.number = ‘360’”. Undoubtedly, it looks like something that will work in a solid way and will not raise an exception. It is an ordinary attribute. But if we use a setter like here: “phone.set_number(‘360’)”, we can see that it can heighten a message about a value error. “inserted phone number is not a phone number.”. In this case, the setter method is a better way to solve this. It simply expresses the code’s beneficial behavior.

Generally, keep away from raising exceptions from your Python properties. You can do that only if you want to use a property to offer attributes that will be only for reading. If you ever need to heighten messages on access or mutation, then you absolutely shouldn’t forget to use getter and setter methods in place of properties.

So in short, using a regular setter and getter can help in solving troubles when they make their code.

In the end, we can wrap up that we know better what is setter and getter methods. But they are not so popular because Python has in-built properties that allow you to mutate and get access to the attributes. But genuinely, as we have before in some situations they have drawbacks that you can fix only by changing properties in regular Python get and set methods.