"Aerial mathematics". Big data in the world of civil aviation. Russian Railways wants to optimize routes using Big Data Big data technology in transport

"Big Data"- a topic that is actively discussed by technology companies. Some of them managed to become disillusioned with big data, others - on the contrary, make the most of it for business ... A fresh analytical review of the domestic and global Big Data market, prepared by the Moscow Exchange in cooperation with IPOboard analysts, shows which trends are most relevant now on the market ... We hope the information will be interesting and useful.

WHAT IS BIG DATA?

Key features

Big Data, today, is one of the key drivers of information technology development. This direction, relatively new for Russian business, has become widespread in Western countries. This is due to the fact that in the era of information technology, especially after the boom of social networks, a significant amount of information began to accumulate for each Internet user, which ultimately gave the development of the Big Data direction.

The term "Big Data" causes a lot of controversy, many believe that it only means the amount of accumulated information, but do not forget about the technical side, this area includes storage technologies, computing, as well as services.

It should be noted that this area includes the processing of a large amount of information, which is difficult to process using traditional methods *.

Below is a comparative table of traditional and Big Data base.

The Big Data sphere is characterized by the following features:
Volume - the volume of the accumulated database is a large amount of information that is laborious to process and store in traditional ways, they require a new approach and improved tools.
Velocity - speed, this sign indicates both the increasing speed of data accumulation (90% of information has been collected over the past 2 years), and the speed of data processing, recently, real-time data processing technologies have become more in demand.
Variety - diversity, i.e. the possibility of simultaneous processing of structured and unstructured multi-format information. The main difference between structured information is that it can be classified. An example of such information would be customer transaction information.
Unstructured information includes video, audio files, free text, information coming from social networks. Today 80% of information is included in the unstructured group. This information needs complex analysis to make it useful for further processing.
Veracity - reliability of data, users began to attach increasing importance to the reliability of available data. For example, Internet companies have a problem of separating actions carried out by a robot and a person on the company's website, which ultimately leads to the difficulty of data analysis.
Value - the value of the accumulated information. Big Data should be useful to the company and bring some value to it. For example, help in improving business processes, reporting or optimizing costs.

If the above 5 conditions are met, the accumulated data volumes can be classified as large.

Areas of Application of Big Data

The sphere of using Big Data technologies is vast. So, with the help of Big Data, you can find out about customer preferences, the effectiveness of marketing campaigns, or conduct a risk analysis. Below are the results of a survey by the IBM Institute on the use of Big Data in companies.

As you can see from the diagram, most companies use Big Data in the field of customer service, the second most popular area is operational efficiency, in the field of risk management Big Data is less common at the moment.

It should also be noted that Big Data is one of the fastest growing areas of information technology, according to statistics, the total amount of received and stored data doubles every 1.2 years.
Between 2012 and 2014, the amount of data transferred monthly by mobile networks grew by 81%. According to Cisco estimates, in 2014 the volume of mobile traffic was 2.5 exabytes (a unit of measurement for the amount of information equal to 10 ^ 18 standard bytes) per month, and already in 2019 it will be equal to 24.3 exabytes.
Thus, Big Data is already a well-established technology area, even despite its relatively young age, which has become widespread in many areas of business and plays an important role in the development of companies.

Big Data Technologies

Technologies used to collect and process Big Data can be divided into 3 groups:

Software;
Equipment;
Service services.

The most common data processing (software) approaches include:
SQL - a structured query language that allows you to work with databases. With the help of SQL, you can create and modify data, and the corresponding database management system deals with the management of the data set.
NoSQL - the term stands for Not Only SQL (not only SQL). It includes a number of approaches aimed at implementing a database that differ from the models used in traditional, relational DBMS. They are convenient to use when the data structure is constantly changing. For example, to collect and store information on social networks.
MapReduce - computation distribution model. Used for parallel computing on very large datasets (petabytes * or more). In the programming interface, not data is transferred to the program for processing, but the program is transferred to the data. Therefore, the request is a separate program. The principle of operation consists in sequential data processing by two methods Map and Reduce. Map fetches preliminary data, Reduce aggregates it.
Hadoop - is used to implement search and contextual mechanisms of high-load sites - Facebook, eBay, Amazon, etc. A distinctive feature is that the system is protected from failure of any of the cluster nodes, since each block has at least one copy of the data on the other node.
SAP HANA Is a high-performance NewSQL platform for data storage and processing. Provides high speed of processing requests. Another distinguishing feature is that SAP HANA simplifies the system landscape by reducing the cost of supporting analytic systems.

Technological equipment includes:

servers;
infrastructure equipment.

Servers include data stores.
Infrastructure equipment includes platform accelerators, uninterruptible power supplies, server console kits, etc.

Service services.
The services include services for building the architecture of the database system, arranging and optimizing the infrastructure, and ensuring the security of data storage.

Software, hardware and services together form complex platforms for storing and analyzing data. Companies such as Microsoft, HP, EMC offer services for the development, deployment and management of Big Data solutions.

Application in industries

Big Data has become widespread across many industries. They are used in healthcare, telecommunications, trade, logistics, financial companies, and government.
Below are some examples of Big Data applications in some of the industries.

Retail
In the databases of retail stores, a lot of information about customers, the inventory management system, and the supply of marketable products can be accumulated. This information can be useful in all areas of the shops.

So, with the help of the accumulated information, you can manage the supply of goods, their storage and sale. Based on the accumulated information, it is possible to predict the demand and supply of goods. Also, the data processing and analysis system can solve other problems of the retailer, for example, to optimize costs or prepare reports.

Financial services
Big Data makes it possible to analyze the creditworthiness of a borrower, and it is also useful for credit scoring * and underwriting **. The introduction of Big Data technologies will reduce the time for consideration of loan applications. With the help of Big Data, you can analyze the transactions of a specific client and offer banking services that are suitable for him.

Telecom
In the telecommunications industry, Big Data is widely used by mobile operators.
Mobile operators, along with financial institutions, have one of the most voluminous databases, which allows them to carry out the most in-depth analysis of the accumulated information.
The main goal of data analysis is to retain existing customers and attract new ones. To do this, companies segment customers, analyze their traffic, and determine the social affiliation of the subscriber.

In addition to using Big Data for marketing purposes, technologies are used to prevent fraudulent financial transactions.

Mining and oil industry
Big Data is used in both mining and processing and marketing. Based on the information received, enterprises can draw conclusions about the effectiveness of field development, track the schedule of overhauls and the condition of equipment, predict the demand for products and prices.

According to a Tech Pro Research survey, Big Data is most prevalent in the telecommunications industry, as well as in engineering, IT, financial and government enterprises. According to the results of this survey, Big Data is less popular in education and healthcare. The survey results are presented below:

Examples of using Big Data in companies

Today Big Data is being actively implemented in foreign companies. Companies such as Nasdaq, Facebook, Google, IBM, VISA, Master Card, Bank of America, HSBC, AT&T, Coca Cola, Starbucks, and Netflix are already using Big Data.

The fields of application of the processed information are varied and vary depending on the industry and the tasks to be performed.
Further examples of the application of Big Data technologies in practice will be presented.

HSBC uses Big Data technologies to combat fraudulent transactions with plastic cards. With the help of Big Data, the company increased the efficiency of the security service 3 times, and the detection of fraudulent incidents - 10 times. The economic effect from the introduction of these technologies exceeded USD 10 million.

Antifraud * VISA allows to automatically calculate fraudulent transactions, the system currently helps prevent fraudulent payments in the amount of USD 2 billion annually.

Supercomputer Watson Company IBM analyzes the flow of data on monetary transactions in real time. According to IBM, Watson increased the number of detected fraudulent transactions by 15%, reduced false positives by 50% and increased the amount of funds protected from such transactions by 60%.

Procter & Gamble with the help of Big Data, they design new products and compose global marketing campaigns. P&G has established dedicated Business Spheres offices where information can be viewed in real time.
Thus, the management of the company was able to instantly test hypotheses and conduct experiments. P&G believes Big Data helps predict company performance.

Office supplies retailer OfficeMax using Big Data technologies, they analyze customer behavior. Big Data analysis allowed us to increase B2B revenue by 13% and reduce costs by USD 400,000 per year.

In the opinion Caterpillar , its distributors lose $ 9 billion to $ 18 billion annually in profits just because they don't implement Big Data technologies. Big Data would allow customers to more efficiently manage their fleet of cars by analyzing information coming from sensors installed on cars.

Today it is already possible to analyze the condition of key components, their degree of wear, manage fuel and maintenance costs.

Luxottica group is a manufacturer of sports glasses such as Ray-Ban, Persol and Oakley. The company uses Big Data technologies to analyze the behavior of potential customers and "smart" SMS marketing. As a result, Big Data Luxottica group allocated more than 100 million of the most valuable customers and increased the effectiveness of the marketing campaign by 10%.

With the help of Yandex Data Factory, game developers World of tanks analyze the behavior of the players. Big Data technologies made it possible to analyze the behavior of 100 thousand World of Tanks players using more than 100 parameters (information about purchases, games, experience, etc.). As a result of the analysis, a forecast of user churn was obtained. This information helps to reduce user leaving and to work with game participants in a targeted manner. The developed model turned out to be 20-30% more efficient than standard tools for analyzing the gaming industry.

German Ministry of Labor uses Big Data in its work related to the analysis of incoming applications for the issuance of unemployment benefits. So, after analyzing the information, it became clear that 20% of benefits were paid undeservedly. With the help of Big Data, the Ministry of Labor has cut costs by 10 billion euros.

Children's Hospital Toronto implemented the Project Artemis project. It is an information system that collects and analyzes data on babies in real time. The system monitors 1260 indicators of the state of each child every second. Project Artemis makes it possible to predict the unstable state of a child and start prevention of diseases in children.

GLOBAL BIG DATA MARKET OVERVIEW

Current state of the world market

In 2014, Big Data, according to Data Collective, became one of the priority areas of investment in the venture capital industry. According to the information portal Computerra, this is due to the fact that developments in this area have begun to bring significant results for their users. Over the past year, the number of companies with implemented projects in the field of big data management has increased by 125%, the market volume has grown by 45% compared to 2013.

Most of the revenue of the Big Data market, according to Wikibon, in 2014 was made up of services, their share was equal to 40% of the total revenue (see the diagram below):

If we consider Big Data for 2014 by subtypes, then the market will look like this:

According to Wikibon, applications and analytics account for 36% of Big Data revenue in 2014 came from Big Data applications and analytics, 17% from computing equipment and 15% from data storage technologies. Least of all revenue was generated by NoSQL technologies, infrastructure equipment and provision of a network of companies (corporate networks).

The most popular are such Big Data technologies as in-memory platforms of SAP, HANA, Oracle, etc. The results of the T-Systems survey showed that they were chosen by 30% of the surveyed companies. The second most popular were NoSQL platforms (18% of users), companies also used analytical platforms from Splunk and Dell, they were chosen by 15% of companies. The least useful for solving Big Data problems, according to the survey results, were Hadoop / MapReduce products.

According to an Accenture survey, more than 50% of companies using Big Data technologies spend 21% to 30% on Big Data.
According to the following analysis by Accenture, 76% of companies believe that these expenses will increase in 2015, and 24% of companies will not change their budget for Big Data technologies. This suggests that in these companies Big Data has already become an established direction of IT, which has become an integral part of the company's development.

The results of the Economist Intelligence Unit survey confirm the positive effect of implementing Big Data. 46% of companies say that they have improved customer service by more than 10% using Big Data technologies, 33% of companies have optimized inventory and improved the productivity of fixed assets, 32% of companies have improved planning processes.

Big data around the world

Today Big Data technologies are most often implemented in US companies, but even now other countries of the world have begun to show interest. In 2014, according to IDC, the countries of Europe, the Middle East, Asia (excluding Japan) and Africa accounted for 45% of the market for software, services and equipment in the field of Big Data.

Also, according to a CIO survey, companies from the Asia-Pacific region are rapidly adopting new solutions in the field of Big Data analysis, secure storage and cloud technologies. Latin America is in second place in terms of the amount of investments in the development of Big Data technologies, ahead of the countries of Europe and the United States.
Next, a description and forecasts of the development of the Big Data market in several countries will be presented.

China
The volume of information in China is 909 exabytes, which is equal to 10% of the total amount of information in the world, by 2020 the amount of information will reach 8060 exabytes, and the share of information in global statistics will also increase, in 5 years it will be 18%. The potential growth of China's Big Data has one of the fastest growing dynamics.

Brazil
At the end of 2014, Brazil accumulated 212 exabytes of information, which is 3% of the global volume. By 2020, the volume of information will grow to 1,600 exabytes, or 4% of the world's information.

India
According to EMC, the volume of accumulated data in India at the end of 2014 is 326 exabytes, which is 5% of the total volume of information. By 2020, the volume of information will grow to 2,800 exabytes, or 6% of the information in the entire world.

Japan
The amount of accumulated data in Japan at the end of 2014 is 495 exabytes, which is 8% of the total amount of information. By 2020, the volume of information will grow to 2,200 exabytes, but the market share of Japan will decrease to 5% of the total information volume of the whole world.
Thus, the size of the Japanese market will decrease by more than 30%.

Germany
According to EMC, the volume of accumulated data in Germany at the end of 2014 is 230 exabytes, which is 4% of the total volume of information in the world. By 2020, the volume of information will grow to 1,100 exabytes, or 2%.
In the German market, a large share of revenue, according to Experton Group forecasts, will be generated by the segment of services, the share of which in 2015 will be 54%, and in 2019 will increase to 59%, the share of software and hardware, on the contrary, will decrease.

Overall, the market will grow from 1.345 billion euros in 2015 to 3.198 billion euros in 2019, with an average growth rate of 24%.
Thus, based on CIO and EMC analytics, it can be concluded that the developing countries of the world in the coming years will become markets for the active development of Big Data technologies.

Main market trends

According to IDG Enterprise, in 2015 companies' spending on Big Data will amount to an average of USD 7.4 million per company, large companies intend to spend about USD 13.8 million, small and medium-sized companies - USD 1.6 million. ...
Most will be invested in areas such as data analysis and visualization and data collection.
In line with current trends and market demand, investments in 2015 will be used to improve data quality, improve planning and forecasting, and increase data processing speed.
Companies in the financial sector, according to Bain Company’s Insights Analysis, will make significant investments, so in 2015 it is planned to spend $ 6.4 billion on Big Data technologies, the average investment growth rate will be 22% until 2020. Internet companies plan to spend $ 2.8 billion, with an average growth rate of 26% for Big Data spending.
When conducting a survey by the Economist Intelligence Unit survey, the priority directions for the development of Big Data in 2014 and in the next 3 years were identified, the distribution of answers is as follows:

According to IDC forecasts, market trends are as follows:

In the next 5 years, the cost of cloud solutions in the field of Big Data technologies will grow 3 times faster than the cost of on-premises solutions. Hybrid storage platforms will be in demand.
The growth of applications using complex and predictive analytics, including machine learning, will accelerate in 2015, the market for such applications will grow 65% faster than applications that do not use predictive analytics.
Media analytics will triple in 2015 and will become a key growth driver for the Big Data technology market.
The trend will accelerate to implement solutions to analyze the continuous flow of information that is applicable to the Internet of Things.
By 2018, 50% of users will interact with cognitive computing services.

Market Drivers and Limiters

IDC experts identified 3 drivers of the Big Data market in 2015:

According to an Accenture survey, data security issues are now the main barrier to the implementation of Big Data technologies, with more than 51% of respondents confirmed that they are worried about ensuring data protection and confidentiality. 47% of companies reported that it was impossible to implement Big Data due to a limited budget, 41% of companies indicated a lack of qualified personnel as a problem.

Wikibon predicts that the size of the Big Data market will grow to $ 38.4 billion in 2015 and will increase by 36% over the previous year. In the coming years, there will be a decline in growth rates to 10% in 2017. Based on these forecasts, the market size in 2020 will be equal to USD 68.7 billion.

The distribution of the global Big Data market by business category will look like this:

As you can see from the diagram, most of the market will be occupied by technologies from the field of improving customer service. Point marketing will be in second place in terms of priority among companies until 2019, in 2020, according to the Heavy Reading forecast, it will give way to solutions to improve operational efficiency.
The segment “customer service improvement” will also have the highest growth rate, an increase of 49% annually.
The market forecast for Big Data subtypes will look like this:

The predominant market share, as can be seen from the diagram, is occupied by professional services, the highest growth rate will be in applications with analytics, their share will grow from the current 12% to 18% in 2020, and the volume of this segment will be equal to USD 12.3 billion. the share of computing equipment, on the contrary, will fall from 20% to 14% and will amount to about $ 9.3 billion in 2020, the cloud technology market will gradually increase and in 2020 will reach $ 6.3 billion, the market share of solutions for data storage, on the contrary, will decrease from 15% in 2014 to 13% in 2020 and in monetary terms will be equal to USD 8.9 billion.
According to the forecast of Bain & Company’s Insights Analysis, the distribution of the Big Data market by industry in 2020 will look like this:

The financial industry will spend $ 6.4 billion on Big Data with an average growth rate of 22% per year;
Internet companies will spend $ 2.8 billion and an average cost growth rate of 26% over the next 5 years;
Public sector costs will be commensurate with the costs of Internet companies, but the growth rate will be lower - 22%;
The telecommunications sector will grow at an average growth rate of 40% to reach USD 1.2 billion in 2020;

Utilities will invest a relatively small amount of US $ 800 million in these technologies, but the growth rate will be one of the highest at 54% annually.
Thus, a large share of the Big Data market in 2020 will be occupied by companies in the financial industry, and the fastest growing sector will be energy.
Following analysts' forecasts, the total market volume will increase in the coming years. Market growth will be ensured by the introduction of Big Data technologies in the developing countries of the world, as can be seen from the graph below.

The projected market size will depend on how developing countries perceive Big Data technologies, whether they are as popular as in developed countries. In 2014, the developing countries of the world accounted for 40% of the accumulated information. EMC predicts that the current market structure, dominated by developed countries, will change in 2017. According to EMC analysts, in 2020 the share of developing countries will be over 60%.
According to Cisco and EMC, the developing countries of the world will be quite active in working with Big Data, largely due to the availability of technologies and the accumulation of a sufficient amount of information to the level of Big Data. The world map on the next page will show the forecast for the increase in the volume and the growth rate of Big Data by region.

ANALYSIS OF THE RUSSIAN MARKET

Current state of the Russian market

According to a study by CNews Analytics and Oracle, the level of maturity of the Russian Big Data market has increased over the past year. Respondents from 108 large enterprises across a wide range of industries showed a higher degree of awareness of these technologies, as well as an established understanding of the potential of such solutions for their business.
As of 2014, according to IDC, 155 exabytes of information have been accumulated in Russia, which is only 1.8% of the world's data. The volume of information by 2020 will reach 980 exabytes and will take 2.2%. Thus, the average growth rate of the volume of information will be 36% per year.
IDC estimates the Russian market at $ 340 million, of which $ 100 million are SAP solutions, approximately $ 240 million are similar solutions from Oracle, IBM, SAS, Microsoft, etc.
The growth rate of the Russian Big Data market is no less than 50% per year.
It is predicted that positive dynamics will continue in this sector of the Russian IT market, even in the context of general stagnation of the economy. This is due to the fact that businesses are still in demand for solutions that improve operational efficiency, as well as optimize costs, improve forecasting accuracy and minimize possible company risks.
The main providers of Big Data services in the Russian market are:

Oracle
Microsoft
Cloudera
Hortonworks
Teradata.

Market overview by industry and experience of using Big Data in companies

According to CNews, in Russia only 10% of companies have started using Big Data technologies, when the share of such companies in the world is about 30%. The readiness for Big Data projects is growing in many sectors of the Russian economy, according to the report from CNews Analytics and Oracle. More than a third of the companies surveyed (37%) have started working with Big Data technologies, among which 20% are already using such solutions, and 17% are beginning to experiment with them. A second third of respondents are currently considering this possibility.

In Russia, Big Data technologies are more popular in the banking sector and telecoms, but they are also in demand in the mining industry, energy, retail, logistics companies and the public sector.
Below we will consider examples of the application of Big Data in Russian realities.

Telecom
Telecom operators have one of the most voluminous databases, which allows them to carry out the most in-depth analysis of the accumulated information.
One of the areas of application of Big Data technology is subscriber loyalty management.
The main goal of data analysis is to retain existing customers and attract new ones. To do this, companies segment customers, analyze their traffic, and determine the social affiliation of the subscriber. In addition to using information for marketing purposes, telecom technologies are used to prevent fraudulent financial transactions.
VimpelCom is one of the striking examples of this industry. The company uses Big Data to improve the quality of service at the level of each subscriber, prepare reporting, analyze data for network development, combat spam and personalize services.

Banks
A significant proportion of Big Data users are specialists from the financial industry. One of the successful experiments was carried out at the Ural Bank for Reconstruction and Development, where the information base was used to analyze clients, the bank began to offer specialized loan offers, deposits and other services. During the year of using these technologies, the company's retail loan portfolio grew by 55%.
Alfa-Bank analyzes information from social networks, processes loan applications, and analyzes the behavior of users of the company's website.
Sberbank also started processing a massive amount of data to segment customers, prevent fraudulent activities, cross-sell and manage risk. In the future, it is planned to improve the service and analyze customer actions in real time.
The All-Russian Regional Development Bank analyzes the behavior of plastic card holders. This makes it possible to identify transactions that are atypical for a particular client, thereby increasing the likelihood of detecting the theft of funds from plastic cards.

Retail
In Russia, Big Data technologies have been implemented by both online and offline trading companies. Today, according to CNews Analytics, Big Data is used by 20% of retailers. 75% of retailers believe Big Data is essential to develop a competitive marketing strategy. According to Hadoop statistics, after the implementation of Big Data technology, profits in trade organizations grow by 7-10%.
M.Video's specialists talk about the improvement in logistics planning after the implementation of SAP HANA, and as a result of its implementation, the preparation of annual reports has been reduced from 10 days to 3, the speed of daily data download has decreased from 3 hours to 30 minutes.
Wikimart uses these technologies to generate recommendations for website visitors.
One of the first offline stores to introduce Big Data analysis in Russia was Lenta. With the help of Big Data, retail began to study information about customers from cashier's receipts. The retailer collects information to generate behavioral models, which enables them to make more informed decisions at the operational and business level.

Oil and gas industry
In this industry, the scope of application of Big Data is quite wide. Big Data technologies can be applied in the extraction of minerals from the subsoil. With their help, you can analyze the production process itself and the most effective ways to extract it, track the drilling process, analyze the quality of raw materials, as well as the processing and marketing of the final product. In Russia, Transneft and Rosneft have already started using these technologies.

State bodies
Countries such as Germany, Australia, Spain, Japan, Brazil and Pakistan are using Big Data technologies to tackle national issues. These technologies help government bodies to more effectively provide services to the population, provide targeted social support.
In Russia, these technologies began to be mastered by such state bodies as the Pension Fund, the Federal Tax Service and the Mandatory Health Insurance Fund. The potential for implementing projects using Big Data is large, these technologies could help improve the quality of services and, as a result, the standard of living of the population.

Logistics and transport
Big Data can also be used by transport companies. With the help of Big Data technologies, it is possible to track the car park, take into account fuel costs, and monitor customer requests.
Russian Railways implemented Big Data technologies together with SAP. These technologies helped to reduce the reporting period by 43.5 times (from 14.5 hours to 20 minutes), and to improve the accuracy of cost allocation by 40 times. Also, Big Data was introduced into the planning and tariff regulation processes. In total, companies use more than 300 systems based on SAP solutions, 4 data centers are involved, and the number of users is 220,000.

Main drivers and market constraints

The drivers for the development of Big Data technologies in the Russian market are:

Increased interest on the part of users to the possibilities of Big Data as a way to increase the company's competitiveness;
Development of methods for processing media files at the global level;
Transfer of servers processing personal information to the territory of Russia, in accordance with the adopted law on the storage and processing of personal data;
Implementation of the sectoral plan for software import substitution. This plan includes state support for domestic software manufacturers, as well as the provision of preferences for domestic IT products when making purchases at public expense.
In the new economic situation, when the dollar rate has almost doubled, there will be a trend towards more and more use of the services of Russian cloud providers than foreign ones.
Creation of technoparks contributing to the development of the information technology market, including the Big Data market;
State program for the implementation of grid systems, which are based on Big Data technologies.

The main barriers to the development of Big Data in the Russian market are:

Ensuring data security and confidentiality;
Lack of qualified personnel;
Lack of accumulated information resources to the level of Big Data in most Russian companies;
Difficulties in introducing new technologies into established information systems of companies;
The high cost of Big Data technologies, which leads to a limited number of enterprises that are able to implement these technologies;
Political and economic uncertainty that led to capital outflow and freezing of investment projects in Russia;
The rise in prices for imported products and a surge in inflation, according to IDC, slow down the development of the entire IT market.

Russian market forecast

As of today, the Russian Big Data market is not as popular as in developed countries. The majority of Russian companies show interest in it, but do not dare to take advantage of their opportunities.
Examples of large companies that have already benefited from Big Data technologies are raising awareness of the power of these technologies.
Analysts are also quite optimistic about the Russian market. IDC believes that the Russian market share will increase over the next 5 years, in contrast to the market in Germany and Japan.
By 2020, the volume of Big Data in Russia will grow from the current 1.8% to 2.2% of the global data volume. The amount of information will grow, according to EMC, from the current 155 exabytes to 980 exabytes in 2020.
At the moment, Russia continues to accumulate the amount of information up to the level of Big Data.
According to a CNews Analytics poll, 44% of surveyed companies work with data no more than 100 terabytes * and only 13% work with volumes above 500 terabytes.

Nevertheless, the Russian market, following the global trends, will grow. As of 2014, IDC estimates the market size at $ 340 million.
The market growth rate in previous years was 50% per year, if it remains at the same level, then in 2018 the market volume will reach USD 1.7 billion. The share of the Russian market in the world market will be about 3%, having increased from the current 1.2%.

The most susceptible industries to using Big Data in Russia are:

Retail and banks, for them, first of all, analysis of the client base, assessment of the effect of marketing campaigns is important;
Telecom - segmentation of the customer base and traffic monetization;
Public sector - accounting, analysis of applications from the population, etc.;
Oil companies - work monitoring and production and sales planning;
Energy companies - creation of intelligent power systems, operational monitoring and forecasting.

In developed countries, Big Data has become widespread in the fields of healthcare, insurance, metallurgy, Internet companies and industrial enterprises, most likely in the near future Russian companies from these areas will also evaluate the effect of implementing Big Data and will adapt these technologies in their industries.
In Russia, as well as in the world, in the near future there will be a trend towards data visualization, analysis of media files and the development of the Internet of Things.
Despite the general stagnation of the economy, analysts predict further growth of the Big Data market in the coming years, primarily due to the fact that the use of Big Data technologies gives its users a competitive advantage in terms of increasing the operational efficiency of the business, attracting additional customers, minimizing risks and implementation of data forecasting technologies.
Thus, we can conclude that the Big Data segment in Russia is at the stage of formation, but the demand for these technologies is increasing every year.

Key Market Analysis Results

World market

At the end of 2014, the Big Data market is characterized by the following parameters:

market size amounted to USD 28.5 billion, an increase of 45% over the previous year;
most of the revenue of the Big Data market was made up of services, their share was equal to 40% in the total revenue;
36% of revenue came from Big Data applications and analytics, 17% from computing equipment and 15% from data storage technologies;
The most popular platforms for solving Big Data problems are in-memory platforms from companies such as SAP, HANA and Oracle.
the number of companies with implemented projects in the field of Big Data management has increased by 125%;

The market forecast for the next years is as follows:

in 2015 the market volume will reach USD 38.4 billion, in 2020 - USD 68.7 billion;
the average growth rate will be 16% annually;
the company's average expenses on Big Data technologies will amount to USD 13.8 million for large companies and USD 1.6 million for small and medium-sized businesses;
technologies will be most prevalent in the areas of customer service and point marketing;
in 2017, the global market structure will change towards a predominance of user companies from developing countries.

Russian market

The Russian Big Data market is at the stage of formation, the results of 2014 are as follows:

market size reached USD 340 million;
the average market growth rate in previous years was 50% annually;
the total amount of accumulated information was 155 exabytes;
10% of Russian companies have started using Big Data technologies;
Big Data technologies were more popular in banking, telecom, Internet companies and retail.

The forecast for the Russian market for the coming years is as follows:

the volume of the Russian market in 2015 will reach USD 500 million, and in 2018 - USD 1.7 billion;
the share of the Russian market in the world will be about 3% in 2018;
the amount of accumulated data in 2020 will be 980 exabytes;
data volume will grow to 2.2% of global data volume in 2020;
the most popular technologies will be data visualization, media file analysis and the Internet of Things.

Based on the results of the analysis, it can be concluded that the Big Data market is still in the early stages of development, and in the near future we will observe its growth and the expansion of the capabilities of these technologies.

Thank you for taking the time to read this voluminous work, subscribe to our blog - we promise many new interesting publications!

In Bashkiria, for the first time, “big data” was used in the analysis of tourist traffic. The State Committee for Tourism of the Republic of Belarus ordered the Ural Center for Monitoring and Analytics to conduct a study based on the dynamics of the movement of mobile phone subscribers.

According to research, from January to November 2018, 1,656 million tourists visited the republic, 60% of whom were men aged 30 to 45, as a rule, employees of commercial organizations, with higher education, with an income of 40 thousand rubles per month. Average length of stay is 3.8 days.

The peak of the tourist flow is in the summer. In June 2018, the number of arrivals was 179 thousand people, in July - 215 thousand people. The minimum indicator was observed in February - 118 thousand people.

Guests came from various regions of Russia. The largest share of visitors - Moscow, Moscow region, Tatarstan - 11% each. Residents of the Orenburg region, Chelyabinsk and Samara regions accounted for the share of tourist traffic in 9%, 7%, 6%. Further, the Sverdlovsk Oblast and the Khanty-Mansi Autonomous Okrug - 3.8% each, the Tyumen Oblast - 3%, the Perm Territory and Udmurtia - each slightly more than 2%.

Foreign tourists came from neighboring countries, as well as India, Spain, Italy, Yemen, Germany, Turkey, Egypt, Nigeria, Israel, USA, Czech Republic, Saudi Arabia, Bulgaria, Iran, China and Finland.

A sociological study was also carried out in the form of surveys of tourists. 37% of the respondents chose a hotel or a hotel for their stay. 17% stayed with friends or relatives, hostels were preferred by 11%. According to the purposes of travel, the tourist flow was distributed as follows: trips to relatives (30%), business tourism (28%), health tourism (18%), excursion (12%), active (8%), pilgrim tourism (0.2%) ...

40% of tourists have come to Bashkiria not for the first time. On the recommendation of friends (colleagues, relatives), 20% came. 24% arrived on a business trip. The least used sources of information when choosing a travel destination for the respondents were Internet portals (3.4%), social networks (1.2%), advertising in the media (0.5%).

In the current 2019, the tourist attractiveness of certain regions of the republic will also be analyzed, the state committee informed.

“Geoanalytics using the capabilities of mobile operators is an advanced method for calculating the tourist flow. At present, only Moscow has such experience, and let me remind you that the latter is in the first place in the national tourist rating in the Volga Federal District, Bashkortostan - the second, ”said Azamat Galin, Deputy Head of the State Committee for Tourism and Entrepreneurship of the Republic of Bashkortostan.

According to the Turstat portal, at the end of 2018, Bashkiria entered the Top 15 ratings of domestic and inbound tourism, taking 13th place with the number of tourists over 2.5 million people (+ 13% to the level of 2017).

These initiatives of the Government of Bashkiria are very interesting and useful for studying tourist flow and planning their activities in order to promote tourism products in the region through the integrated provision of services to tourists, including using it-technologies.

By the way, Nizhniy Nogorod is mentioned in the news. We previously reported that the "Guest Card" project has been implemented in this city, by which it will be possible to track the movement of tourists visiting the city's attractions, their interests, tourists will be able to receive various discounts, as well as use public transport for free.

All these initiatives are being implemented in the regions isolated and singular, without federal participation.

WHAT TO SPEAK ABOUT?

The essence of the matter is that the issue of using electronic visas for foreign citizens arriving in the Russian Federation is currently being resolved. According to the Tourism Safety Association, the use of such visas using special digital technologies without integrating the system of migration and registration of tourists in hotels and the above-mentioned services on the "guest card" does not make sense. This is not a government approach.

In our opinion, a systemic, state approach should include taking into account all these elements. A tourist must register at the borders once, having received an electronic tag, and then move around the country, register at hotels (already without migration registration), visit museums without problems, receive various discounts, use public transport for free or with discounts. And at the same time, this approach will allow - both to ensure national security by recording the movements of foreigners, and to free hoteliers from the headaches of registration and migration registration, and to the tourism authorities in the constituent entities of the Russian Federation to receive information about the most popular objects of the region (city) and, on its basis, form tourist offers, thereby getting the maximum benefit.

AND FOR THIS ALREADY IS EVERYTHING!

Namely, the Decree of the Government of the Russian Federation dated August 6, 2015 No. 813, which approved the Regulation on the state system of migration and registration registration, the implementation of which can significantly affect the hospitality and increase the inbound tourist flow in general. This is exactly what the Chairman of the Board of the Association "Tourism Safety" spoke about on December 6, 2018 in the Federation Council. Sergey Gruzd to the participants of the round table on the topic "Topical issues of the application of electronic visas for foreign citizens arriving in the Russian Federation, and improving the legislation of the Russian Federation in this area"

Recall that the issues of improving migration and registration accounting, simplifying the visa regime, developing and implementing a single biometric identifier for travel will be the subject of discussion within the framework of International Forum "Tourism Safety" - TSIF - 2019.This Forum is a key professional event, where representatives of government authorities, professional community and business on one platform discuss topical issues of ensuring the safety of tourism. The format of the Forum includes 4 breakout sessions.

10/01/2018, Mon, 10:03, Moscow time , Text: Maria Sysoykina

The Safe Transport Innovation Center, established a year ago as part of the Moscow Metro, brings together developers of solutions for working with digital technologies. As part of the first strategic session of the innovation center, a discussion was held on new technologies offered by Russian companies, as well as initiatives already implemented by the center.

Community around "Safe Transport"

The Safe Transport Innovation Center began creating a community of experts and developers to exchange ideas and experience on the use of modern technologies in solving various transport problems for Moscow. The community will bring together both those companies that are already working with Safe Transport and new members. At the first strategic session of the center, representatives of ABBYY, Maxima Telecom, Yandex.Taxi, Avito, Software Product and others shared their vision of the necessary technological changes in transport in Moscow, discussed the role of technology in the formation of new innovative services and offered ideas for personalizing the interaction of the city with its inhabitants.

Big data is changing communications

The idea of creating the center was born in August 2018.The main goal of this initiative is to transform interaction with passengers, bring communications with citizens to a new, personalized level. Big data analysis helps you achieve your goals. The innovation center has the ability to work with the data of organizations subordinate to the Department of Transport, performing its research, testing hypotheses, doing work on building segments for targeted communication companies.

“We collect a lot of heterogeneous depersonalized data about passengers and, based on the analysis, we can provide targeted information to citizens of important information,” explains the head of the Innovation Center. Yuri Emelyanov... - Scenarios can be very different. For example, there are often route changes, repairs, traffic blockages in connection with some events, activities. By analyzing the data, we can inform about changes in a personalized manner to those passengers who often travel on these routes. "

Innovation center projects

There are also more ambitious projects in the Center's treasury, for example, an analysis of the satisfaction of Moscow districts in the use of ground transport. Center experts conducted numerous surveys on this topic, analyzed the results and formulated initiatives to change routes, schedules and stops based on the results. The Center submits these initiatives to various steering committees held within the transport complex and, if approved, are implemented by subordinate organizations. Feedback on the implemented initiatives is again sent to the innovation center, where the results of work and the degree of satisfaction of citizens are assessed. This program started in March 2018 and has shown itself to be quite successful so far. The Center is now actively participating in a similar program for the Moscow Metro.

Of particular interest is the project of the Center for analytical support of events within the framework of the FIFA World Cup. The center's experts analyzed passenger traffic on the days of matches held at Moscow stadiums (Luzhniki, Spartak, fan zone on Vorobyovy Gory), implemented satisfaction surveys as soon as possible after the game and developed recommendations to optimize the load on the city's transport system and make it more efficient. organization of transport services.

Assessment of the load distribution on the Luzhniki stadium. Fragment of the analytical report on the match Russia - Saudi Arabia, which took place on June 14, on the opening day of the championship

Support of mobile applications for citizens has become a separate area of work of the center. Safe Transport cooperates with a number of developers, including Infocompass, which is developing the Moscow Assistant application. “We are trying to support initiatives to create various services based on mobile applications for citizens. For us, this is one of the channels of communication with the population of the city, - says Yuri Yemelyanov. “For example, the experts of the Center, together with the developers of the Moscow Assistant mobile application, are working to improve the algorithm for recognizing the state registration plate.” The Innovation Center has many ambitious targets for the upcoming 2019.

Data has become an important asset, and is of great value in and of itself. With the right approach to determining the owner and carefully building access to them, they can bring profit to all participants in the transportation process. But they can also become a bone of contention, - the magazine writes.

“Data has become an asset. Data today is 21st century gold and oil. The one who quickly learns to work with them, process, cluster, make products that increase added value out of them, will be ahead, ”Mikhail Mishustin, head of the Federal Tax Service, convinced his listeners at the session“ Digital transformation and quality of life. A View from the Regions ”, which took place in the framework of the Russian Investment Forum in Sochi. He is talking about the so-called big data - and who else but the head of the Federal Tax Service, where data on the income and property of millions of Russians is collected, understand their full value? But in fact, the official only repeated a phrase that can now be heard in hundreds of forums around the world from the heads of thousands of companies, including global ones. And the first question that arises: since big data has become a valuable asset, then there should be rules that describe how to handle it, who owns it, and at what price can this data be bought?

Big data technology implies the presence of three elements: huge amounts of data, computing power for very fast processing of this data and special mathematical models that allow comparing predetermined parameters, access to which was previously prohibited. This allows you to identify new, very often unobvious connections and patterns and already on the basis of them make management decisions and make a profit (or, as an option, solve socially important problems).

In order to benefit from big data, technologies had to mature. More recently, companies have at their disposal computing power and algorithms that are able to quickly process huge amounts of data in real time, data centers where this data can be stored, the so-called Internet of Things is developing, which allows you to receive data from equipment in real time. and various devices, the characteristics are improving and the price of sensors that are used to collect data is falling.

Alexey Fedoseev, Head of Customer Service Department at Siemens Mobility, defines the boundary at which data can be considered large: “1 million measurements, so-called data points. From now on, we can implement analytical models that are based on the Big Data approach. "

The pioneers were aircraft manufacturers. The value of big data, which can be used to predict equipment malfunctions and failures, is especially great in this industry. For example, now a Boeing 737 with two engines transmits 240 thousand terabytes of data in six hours of flight (the amount of data on paper in the Lenin Library is more, but not much - about 84 times). We are talking about the removal of several hundred thousand parameters per flight, although previous generations of aircraft collected only a few hundred of them.

Last year, the head of the mining company Tinto (in its fleet, data is taken from unmanned dump trucks, drilling in quarries, locomotives and in the port) said that the central control center in Perth receives 2.4 terabytes of data every minute (approximately 3.5 thousand tons). terabyte per day).

Andrey Borodin, chief project engineer at the Design and Technology Bureau, Center for Digital Technologies of the Department of Informatization of Russian Railways, says that, from the point of view of professionals, data can be hot (that is, it is processed immediately, in real time), warm and cold (unused, but left for storage).

“And even raw data are not unreasonably viewed by many companies as an asset that can bring value, even if companies cannot use it now - to make predictive models or response systems in real time,” says Oleg Pyatakov, head of investment analysis at the company. 2050. digital ". He is confident that generating data for the sake of data is counterproductive, at least in the short term: “We need the ability to link data to each other (device / user identifiers, time stamps), at least the minimum data significance for those target parameters that we are trying to optimize, the ability to develop a control action ... Indeed, in traditional (old) management systems, the norm was a situation when more than 95% of the collected data, for various reasons, were not used to make a decision. "

Russian Railways became one of the first Russian companies to embark on the digital transformation process. And of course, the holding also works with big data technology. Naturally, the first area for their application is obvious - the regular collection of data from rolling stock and infrastructure using the Internet of Things.

At Siemens Mobility, which is a strategic partner of Russian Railways in this area, a clear distinction is made between two concepts - data and information. The data generated by the rolling stock and infrastructure, according to Alexei Fedoseyev, belongs to the operating organization: “As soon as we delivered the technical systems to Deutsche Bahn or Russian Railways, the data belongs to them.”

Then, within the framework of service contracts, within the framework of separate contracts for the processing of this data, they are converted into useful information. For example, the Lastochka trains operated at the MCC generate diagnostic messages about the technical condition of individual electric train subsystems. This data is aggregated and transmitted via a secure channel to a server on the territory of the Russian Federation. And only then, says Aleksey Fedoseev, in the Center for Analysis and Data Processing, created jointly by Russian Railways and Siemens in February 2017, will these aggregated data be converted into useful information.

The center's employees use analytical models that, on the basis of the obtained technical parameters, make it possible to implement the concept of predictive maintenance, predict failures of critical rolling stock units, the expert says. An example is the processing of data received from a traction drive system. But not only. For example, the system of passenger doors is also monitored. When driving in the city train mode, the operation of the passenger door can affect the time the train spends at the station, failures and failures in their work can affect the violation of traffic schedules. Employees of the repair department of the Directorate of high-speed traffic of Russian Railways have access to this information through the Cormap computerized maintenance system. The system is open, on its basis decisions are made on the issuance of trains to the line.

Predictive analytics models for the operation of high-speed trains supplied by Siemens for German, Spanish, Russian, Turkish railways, as well as Eurostar, have been improved over the past three to four years. The more data is processed, the more accurately the models function. The result is an increase in the technical readiness of trains. For example, the work of the Siemens Remote Monitoring Center on Velaro trains in Spain began a little earlier than with Sapsans in Russia. The models make it possible to predict traction motor failures in five to seven days, which has led to the almost complete exclusion of the possibility of violation of the travel schedule due to a decrease in thrust. As a result, RENFE has demonstrated its readiness to compensate passengers 100% of the ticket price if the train is delayed by more than 15 minutes on the Madrid - Barcelona line. The reaction of passengers was not long in coming: the share of rail traffic in passenger traffic in this direction increased from 20 to 61%, and air traffic decreased from 80 to 39%.

If we take the Russian experience in the implementation of similar models of predictive diagnostics of Sapsan trains, then, according to Alexei Fedoseev, the positive effects are obvious: on the Moscow - St. Petersburg line, the Sapsan train fleet has already covered more than 7 million km without delays due to technical failures. that exceed 5 minutes (this is one of the parameters that the company uses to assess the level of reliability).

An important part of working with big data was the creation of a so-called trusted environment - it is designed for the safe use of data, excluding unauthorized access to it. For example, the "Trusted environment of the locomotive complex" is built to access data that will be generated by locomotives, consumers of this data - employees of the Russian Railways holding, service companies, rolling stock manufacturers and component manufacturers.

Relationships are not always based on partnerships. In this case, a confrontation between the parties involved in the provision and processing of data is possible. How this can happen is demonstrated by the story that is currently developing with the Danish company Maersk, the leader in ocean shipping. Back in 2014, the company decided it would digitalize its ocean shipping business. Maersk then reported that a simple sea shipment of chilled fruits from East Africa to Europe goes through a chain of 30 people and organizations and requires about 200 acts of interaction (transfer of documents, communication) between them, and 20% of the cost of shipping a batch of goods goes to processing. transfer of documents and administration of the process. Maersk was going to drastically cut costs in this area, where no major changes have occurred in 60 years.

In 2016, she decided on a technology and a partner, began cooperation with IBM companies as a carrier of advanced knowledge in the blockchain. The blockchain smart contract system, called TradeLens, began testing in 2017. In January 2018, Maersk and IBM announced a joint venture. We worked with partners to figure out how to speed up the transfer of information and reduce the number of errors. It was announced that a full commercial version of TradeLens will be launched by the end of 2018. By mid-2018, the system contained data on 154 million events (dates of arrival of ships, reports on the dispatch and arrival of containers, customs permits, commercial invoices and bills of lading, that is, documents on the acceptance of cargo by the carrier from the shipper), their number increased by 1 million each day - in general, TradeLens was ready for full operation.

At the test stage, 92 participants joined the system: shipowners, ocean carriers, shippers, ports (for example, the very large port of Rotterdam, through which up to 2/3 of ocean cargo for Europe passes) and customs. But at the same time, as testing ended, it became known that other ocean carriers flatly refused to connect to TradeLens. And without the information of these players, the full use of the system is excluded.

It looks like Maersk came as a surprise to this resistance. In mid-November, the Danish company accepted an offer from competitors in the top six (MSC, CMA CGM, Hapag-Lloydand and Ocean Network Express) to join a non-profit association that will develop new standards for information exchange in the industry. André Simcha, CIO of MSC, the No. 2 ocean carrier, told reporters that his company would be happy to join TradeLens if the company becomes more open. In general, MSC likes the idea of working through a non-profit association much more, because, despite the promises of equal access to information, all intellectual rights to TradeLens are divided between IBM and Maersk. The carriers did not like the prospect of giving their data to the system, while their main competitor would be making money on it. Oleg Pyatakov nevertheless believes that Maersk has chosen the right path and in the end the proprietary solutions of powerful companies will win, and open standards without the participation of strong players will give way. But Maersk will have to fight to own a valuable asset like data. In November, a rival system with TradeLens was announced.