What were the first search engines? Search engine Which search engine appeared first
Hello! Today there will be an article about the search engines of the world, which ones generally exist, when they appeared, what drives specifically in Russia, and what is on the entire globe. The article turned out to be huge, so sit down comfortably, preferably having prepared a thermos with tea in advance 🙂
In order to become an expert in any field, life hacks, no matter how useful they may be, are not enough. It is necessary to thoroughly understand the issue and constantly expand your horizons.
Have you thought about how the things we use every day came about? What are the search engines besides Google and Yandex? And why do all SEO studios and people choose the latter? Yes, but didn't find the answers? They are in this article.
A long time ago, when everything was just beginning on the Internet…
In order to put things in order at least a little, the directories Yahoo, DMOZ, etc. were developed. By the way, some of them have not sunk into oblivion so far. A directory is something like a library. In it, you can not only save, but also divide links to sites into groups.
The World Wide Web grew at an incredible rate and catalogs no longer met the needs of visitors. Then the programmers began to work on searching within directories, and then on the first search engine.
Wandex is a pioneering search engine
The search engine that has become a pioneer is Wandex (no, this is not the "father" of Yandex, there are no "family" ties between them). She, like all other search engines, was so far from ideal that she gave out literally everything, and not what most suited the request. There were no ranking algorithms.
The first system that was created to search for information on the Internet was Wandex, it no longer exists. It was created by World Wide Web Wanderer, a Perl bot developed by Matthew Gray of the Massachusetts Institute of Technology in 1993. He has also been involved in the creation of Apache, the development of the MIT website, and other projects. Today Matthew Gray is an employee of Google.
In 1993, there were only 623 sites on the entire Internet, so Vandex had less work than modern search engines. Moreover, thanks to such modest measures, powerful servers with multi-stage clustering, complex work of search algorithms and ways to optimize them were not needed to create indexes and display results.
Wandex and other early services had an imperfect search engine. When requesting a word, they displayed not the most relevant links, but all pages, their data, without analysis and ranking. Therefore, for some time, the real competitors of search engines were page directories - web sites with manually selected categories of links to web sources.
Vandex restoration
December 31, 2011 it became known about the revival of this system. On January 1, 2012, the World Wide Web Wanderer search bot was registered by the owners of many sites, which caused some discussion. Today on the main page there is a search bar with the inscription Wandex.
The letter X is depicted as crossed lines, one of which has the shape of a magnifying glass - a frequently used symbol in a search engine.
The most popular Internet search engines in the world and Russia
Even a 1st grade student can name the most popular search engines. These are Google and Yandex. You can argue day and night about which one is better. But why? They are different, and which one to enter depends on your goals. For you, I have prepared a large list of 16 search engines, these are almost all the search engines that currently exist or existed before.
Let's start with search engines in Russian and the most popular in Russia, and then move on to the world ones.
Yandex
If you believe what they write in LiveInternet, then Yandex is preferred by the majority of Russians. However, the difference between him and Google is only one percent.
Among those involved in the creation and SEO promotion of sites, there is an opinion that Yandex is sharpened for commercial requests, that it has several times more of them than a competitor. In my opinion, this is another speculation. As practice shows, regionality determines the size and structure of the target audience.
The history of Yandex's emergence began in the late 1980s, with the creation of search technologies at Arcadia, which later became part of the CompTek group. The creators came up with several search programs - the International Classifier of Inventions and a Bible search that takes into account the morphology of the Russian language. The search method is called Yandex.
The first active version of the search application called Yandex appeared in 1993, in 1995 the computers of the company that created the search engine were connected to the Internet, and in 1996 work began to refine search methods.
The official birthday of the Yandex.ru search engine is September 23, 1997. On this day, the system was presented at the Softool exhibition in Moscow. Yandex in 1997 took into account the morphology of the Russian language, the distance between words, and was able to distribute documents based on their relevance to a given query.
In 1998, contextual advertising appeared on the portal for the first time - advertising that meets the request of a potential user. With this method of display, advertising is seen only by the target audience. Contextual advertising has become one of the important answers to user questions, and in the future - the main business model of Yandex.
Yandex was founded in 2000. Its founders were the shareholders of CompTek. The organization ru-Net Holdings invested more than $5 million in Yandex, its profit was 35%. Shareholders included Yandex managers and its chief developers of the search engine.
Arkady Volozh became the CEO of the company, and Ilya Segalovich became the director - he was also responsible for the implementation and development of technologies. The Yandex team consisted of 25 people, and all the information was on one server.
In 2013, the Russian search engine Yandex took fourth place in the world in terms of the number of requests processed, overtaking Microsoft and losing the leading positions to Google, Yahoo and Baidu. Yandex search revenue in the domestic market is 61.6%. The number of international audience of the portal is 93.1 million people. The company's net income for 2013 increased by 80%.
Google rules the ball in all countries of the world, except Russia. He came to us relatively recently, in 2004, when Yandex was already on its feet.
Google has spread so much that the word "google" has become a household word, they mean searching for something on the Internet. It's funny, but when I ask my mother to "google" something, she still opens Yandex. Maybe he doesn't know what other search engines are.
Google first appeared as a development of the scientific project of Larry Page and Sergey Brin. In 1996, Stanford students worked on the Digital Library Project (SDLP). It was the goal of developing new technologies for a common, integrated and universal digital library. Funding was provided by the National Science Foundation.
To find a dissertation topic, Page considered, among other information, the study of the mathematical properties of the Internet, representing the structure of links in the form of a large graph. His supervisor, Terry Winograd, encouraged him to go with this idea (Page later commented that it was the best advice he'd ever received), and he completely immersed himself in the problem of figuring out which sites were linking to that page, considering the number and the nature of the backlinks.
He was soon joined by Stanford graduate student Sergei Brin, who was Page's best friend. Together they created the PageRank algorithm. After identifying BackRub results, which consisted of a list of URLs sorted by the number of backlinks to that URL, they realized that a PageRank-based search engine would bring them high results.
This technology of page ranking and rating has already been studied by Robin Lee, who created a small search engine RankDex in 1996. This technology was patented and applied later after the founding of Baidu in China.
The first investment in Google was recorded in August 1998 in the amount of 100 thousand dollars from Andy Bechtolstein, one of the developers of Sun Microsystems. The company was officially registered on September 4, 1998 in a friend's garage in Menlo Park, California. The name Google comes from the word "googol", meaning the number 1 and one hundred zeros. At the end of 1998, Google indexed over 60 million pages.
The homepage was still BETA, but the search engine was already receiving positive feedback: Google search scores were higher than other systems, high technological innovation than overloaded web portals.
Google search engine attracts a huge number of Internet users with its minimalism and accessible interface. In 2000, Google began selling ads related to key search queries. The ad consisted of text, so it did not load the page design and did not slow down its loading speed.
Key queries were implemented based on a combination of the cost of the offer and the number of clicks, the cost started from $0.05 per click. This keyword advertising model was first proposed by goto.com. Unlike its competitors, Google has managed to break into a new Internet market.
Those who are in the shadows
If you, like my mother, do not know what other search engines are, then look. A lot of them!
The share of Mail.ru is small, but it is gradually growing. First of all, at the expense of mail, Odnoklassniki and other services from this search engine.
The history of Mail.ru began in 1998, when a group of programmers from St. Petersburg developed the mail.ru mail service and sent it to the Internet for testing. In May 2005, mail.ru Group was established. It promotes a common integrated platform for communication and entertainment Internet services.
The company owns a leading mail service, one of the most visited sites on the Runet, popular social networks - Vkontakte and Odnoklassniki, major games Warface, Armored Warfare, Skyforge and Perfect World, the maps.me service - offline maps and navigation for mobile gadgets based on OpenStreetMap , food delivery service Delivery Club, a trading place for Chinese products Pandao.
Rambler
Rambler is one of the oldest search engines. It could become the most popular in Runet, but it did not work out. Now this search engine is not a search engine in general, but several services that are made on the Yandex engine.
Rambler's attendance is the envy of newcomers - it exceeds one million people a day.
Rambler has a lite version called Rambler Lite. Everything is the same in it, only there are no annoying ads, news and weather.
In 1996, the Rambler search server was first created. Therefore, the year the search engine was opened is this date. October 8, after successfully passing the test, Rambler began to function as a search engine.
This name of the search engine arose not by chance. When translated from English, it means a tramp, which accurately reveals the method of the search engine. It surfs the Internet endlessly and independently collects data on all the documents it finds.
In 1997, the Rambler's Top100 rating classifier appeared for the first time, which allows, on the basis of objective information, to assess the popularity of a web resource. The appearance of such a counter contributed to the revival among webmasters, who began to pay more attention to the quality of the sites being developed.
Despite the fact that Rambler is considered the oldest Internet search engine, it is constantly being upgraded. New versions of the search engine can greatly increase the speed of the search. The search index is updated every day, which allows you to find the latest news.
Rambler is constantly being upgraded and remains one of the most popular search engines. While Internet users will need to find the right information, this system will roam the Internet and collect important information.
In 2011, Rambler completely abandoned the promotion and support of its own search. He entered into an agreement with Yandex. Since then, it has been working in conjunction with Yandex and is no longer an independent search engine.
Nowadays it is not easy to surprise, but this search engine has succeeded. Nigma can solve problems in mathematics and chemistry, and also offers searches for music, games, books, and torrents.
Nigma is an intelligent search engine developed by Russian developers Vladimir Chernyshev and Viktor Lavronenko. It all started with their meeting in 2004, and then fruitful cooperation began to create a high-quality new project for Runet.
Despite the fact that the service uses the sources of the most popular search engines when issuing, it has a lot of interesting ideas.
How did the search from Nygma come about?
This search engine searches not only for its own index, but also for the indexes of its colleagues - Yandex, Yahoo, Rambler, Bing. As for Google, in 2007 Stanford University professor Hector Garcia-Molina, who at one time was the scientific supervisor of the founders of this most popular search engine, took part in the program for creating algorithms for Nigma.
As of February 2009, Nigma had more than 7 billion documents in the sum of indexes with involved search engines. This is the first runet clustering system, but let's figure it out together.
When you enter a query in a search engine, Nigma generates a list of documents, dividing it into clusters. Each of their set can be rejected, then the search will be performed with high relevance. For example, requests received from online stores create a separate cluster - if you are not interested in such documents, then simply exclude them from the results.
Nigma service features: mathematics, chemistry, music
Nigma.ru has its own unique characteristics, for example, it uses smart search suggestions. To orient, the search engine issues them even at the time of entering the request itself. They can be conditionally divided into three groups: short, precise queries and encyclopedic reference.
Nigma helps you quickly find torrent trackers without registration and advertising banners. Just provide the title of the movie and the word torrent for that. You can instead specify to download the movie with its title and also get links to torrent sources. This is a very interesting specialization, with which you can not get lost in the world of cinema.
Nigma still has more characteristic features. But let's take a closer look at her specialization in solving various problems in mathematics, chemistry, physics - this is a universal assistant for schoolchildren.
Nygma Mathematics
This is a service with which you can solve various mathematical problems online and reproduce calculations - this is a simplification of expressions and even the solution of systems of equations. To do this, you need to specify them as a simple query in the search bar.
The Nigma system also recognizes a huge number of different mathematical and physical constants, units of measurement - the result of actions with different values is obtained. Another possibility is solving equations with mathematical expressions, constants can be specified both in abbreviated and in full form.
Nigma Chemistry
Not everyone can solve problems in chemistry, but they also need to be checked. It is better to use Nygma than to write off answers from reshebnikov. In any case, Nigma-chemistry is a cool service for self-education in this direction. In the corresponding category, you can find an introductory video to understand how to work with the system.
You can search for individual chemicals, specify either short symbols or just the name of the letters. Nigma controls the process - for example, if you mistakenly specify a reaction formula that does not exist at all, the system will definitely notify you about this.
Nygma music
The search engine also has a multimedia section dedicated to searching for music and listening to it online. You can search by song title, artist, or album. Before downloading music, the system offers to listen to it.
Additional features of Nigma
This was not the whole list of features of the Nigma search engine, some categories are of particular interest:
- Official web resources. nigma.rf increases the priority in the results of primary data sources. That is, if you want to know something about the person who has his own website, or about the company, then such information will be displayed in the forefront.
- English prompt. This service checks the compatibility and frequency of words used in English expressions. You specify any expression in the search string, if it is incorrect, the system will offer the correct options.
- Library search. To do this, the service has a special category Books. You can find any information on authors, texts of works.
- Deciphering abbreviations. The smart system recognizes not only Russian, but also foreign abbreviations. Just indicate in the search engine what you are interested in, in addition to the results on the left, you will see options.
- Table search. You can automatically retrieve information about objects in the form of tables. The search engine was the first among competitors to offer such a feature.
- Spam control. This service offers users the opportunity to help develop the search engine. By simply using the search bar and browsing the sites, you can notify the moderator that one of them is unreliable.
Nigma is an interesting search engine with its own personality, despite the use of indexes from other search engines. There are a lot of different useful functions here, most importantly, everything is sorted and specific.
The last search engine on the list. And for good reason. Do not under any circumstances use it. Don't believe me, read the reviews. Many had to pull this search engine out of their computer like a tick. Alas, but it is. All that people are interested in regarding Webalta is the removal methods.
Vebalta is a Russian search engine. It is promoted using technologies that allow, without the knowledge and consent of users, to set the site start.webalta.ru as the home page in the browser on their computers, as well as as the default search engine.
Also, an additional parameter for launching the Webalt site is added to the browser launch commands in the properties of the corresponding shortcuts. The company is suspected of links to cybercrime.
At first, Vebalta was an ambitious project that developed rapidly thanks to a company of enthusiasts without investing money in it and without attracting third-party investments. The development plans were impressive and consisted of real competition with Google and Yandex, which by that time were firmly on their feet.
Naturally, this was a utopia, which ended with the sale of the project to those who today made it into a frank GS that attracts traffic in non-kosher ways in order to receive income from teasers from the Market Guide and other advertising. By itself, such a method of income has a right to exist, but it does not suit a site that positions itself as a search engine.
Google, for example, still adheres to the principle that the main page of both Google.com and google.ru should have nothing but a query string. Its main and more successful competitor in the Runet search market, on the contrary, uses the main page to the maximum - both for advertising and for the needs of users.
You can customize the Yandex home page, add widgets, and change themes. But none of the self-respecting search engines fell to the theses, except for Vebalt. Therefore, at the project launch stage, Vebalta even had its own contextual advertising system under the telling name Optimist.
But for the search engine, launched in mid-2000, to be able to compete with those who entered this market 10 years earlier and firmly settled in it, a miracle had to happen. Even those who started in the mid-nineties on a par with the leaders did not manage to stay in the leading positions - there is no Aport search engine at all, and the Rambler search completely switched to Yandex.
Even a monster like Yahoo is no longer considered a search engine, but uses Bing's capabilities for this purpose.
The developer of Vebalta once wrote that they had the opportunity to get a solid investment package, but it was given to another promising startup from the same niche - the smart search engine Nigma, which is still in demand, even with such little traffic as Vebalt, but she did not stoop to a viral introduction into users' browsers and does not make money on teasers.
Before Vebalta was sold, there were rumors that one of the most impressive data centers at that time was created for it, which was later given over to the creation of bulletproof hosting.
From 2008 to 2010, the site of this search engine partially did not perform a search at all, and since 2010 a viral epic has appeared, which, although it helped to increase traffic, completely buried Vebalt's reputation.
bing
Bing changed not one, not two, or even three names. But it doesn't matter, because it is close to Google in quality.
On June 1st Microsoft Corporation created its first search engine and called it Bing. Spiteful critics began to rub their hands maliciously, enjoying another failure, but this system went uphill. For a while, the search engine competed with Yahoo for a long time. But Yandex and Google quickly overshadowed it due to the fact that the Kumo engine does not index sites in Russian well.
Today, Bing is the default search in Internet Explorer and Mozilla Firefox Internet browsers, as well as on mobile gadgets with the Windows operating system. It is the second most popular search engine in the world. Despite the weak activity of users in Russia and the CIS countries, part of the traffic around the world is more than 20% (the main users are residents of America, China and Europe).
If we talk about demand among the Russian-speaking audience, here the Bing search engine is not very successful. According to statistics for September 2018, part of the traffic in Runet is 0.28%, which is very small. Users pay more attention to Yandex and Google.
This is due to the fact that Bing does not officially work in Russia. For Runet users, only the Russian-language version of the site is provided, which provides results relevant to English-language resources. That is why Russian smartphones with Windows OS use Yandex.
Bing features
The Bing search engine has a number of unique features that make it stand out from the rest of the search engines. Due to the constant competition with Google, the system is constantly updated and provides new features, among which are the following:
- Each user can choose the format for displaying snippets in the search results. For example, a full or abbreviated description, and so on.
- The ability to choose the number of displayed web resources on one issue page, as well as the option to launch new sites.
- Interaction with Microsoft Office services. For example, the built-in search for word meanings in Microsoft Word.
- A search log that lists all queries entered from this browser.
- The ability to save places on the map, pictures, videos and news in My collections.
In addition to the above, the Bing search engine has basic capabilities and tools inherent in other PSs.
Service for placing contextual advertising - Bing Ads
Of course, among webmasters from Russia, this service is not as popular as the same Yandex.Direct. The use of Bing Ads is reasonable only in situations where the promotion of the project is required outside the Runet.
Due to the least popular search engine Bing, there is little competition here, while the quality of traffic is not inferior even to Google, and the audience's solvency is at the same level. Promotion of Internet projects in Bing does not require specific knowledge and skills.
Some ranking factors are different from those that are suitable for Google, and sometimes even considered the complete opposite. Sometimes what works well for a website in one search engine may be harmful in another.
For example, if in Bing the exact occurrence of keywords in headings, subheadings and the text itself is welcomed, then in Google this can help to fall under the filters.
Different types of issuance
Like other search engines, Bing can find not only web resources with relevant content, but also pictures, news, and videos. Everything is familiar here for Google and Yandex users. You can select the required section above the results of the issuance.
Maps from Bing
This tool allows you to find specific places on the map, companies, restaurants, medical centers, and so on. To use it, just click on the link https://www.bing.com/maps/ and enter the desired address or company name in the search bar.
You can view maps in two modes: roads and hybrid view. You can also enable full screen mode. The Routes button allows you to continue following from point A to point B, it will show the approximate time and length in kilometers. The found path can be printed. If you're signed in to Bing Search, you'll be able to save places to My Collections.
I have been promoting informational and commercial websites since 2013.
Translator powered by Microsoft Translator
This is another useful tool that allows you to get a reliable translation with the ability to vote for one or another result. The service has more than 60 languages at its disposal. Bing Translator also includes a mobile app, business and education tools.
The Bing search engine from Microsoft deserves the attention of users. It has a number of amazing features and benefits, as well as huge potential. Of course, it is difficult to get ahead of such a giant as Google, but recently the part of the traffic that falls on Bing has increased markedly, which means that the developers are moving in the right direction.
Yahoo
Officially, Yahoo is not a search engine. The fact is that according to the deal concluded between Yahoo and Bing, all Yahoo sites now have the Bing engine.
According to popular web ratings, Yahoo is America's oldie of the Internet, with over 700 million monthly users. In America, the love for the Yahoo search engine among Americans is higher than for Google. The service was first created by David Filo, an electrical engineering student.
In 1994, the developers made a catalog with links to other web resources. Since then it was extremely difficult for Bati sites on the World Wide Web, the directory became popular in a short time, and by the fall of 1994 it was used by more than 100 thousand users, and it had more than one million requests.
History of Yahoo
In 1995, Yahoo Corporation was established, and a year later branches were opened in countries such as Germany, Italy, Sweden, China, India, Japan and Canada. Then there was the Internet boom, and at that time the search engine made great money on advertising.
By that time, the major search engines Lycos, MSN, Excite and Yahoo had appeared. They developed and grew rapidly. And in order to attract and retain more users on web resources, search engines created more and more services:
- Yahoo! Mail. Among the advantages of mail from Yahu, it is necessary to highlight comfortable functionality, spam filtering, the use of AJAX technology for the Russian-language interface, a large mailbox size (up to 1GB), the ability to use RSS and synchronization with various web applications. The disadvantages of this service are a large amount of advertising, the inability to apply tags to sort messages, as well as some shortcomings in the Calendar and Notepad categories.
- Yahoo! games". This section is based on com developments acquired at the very beginning of the company's activities. Today on this service you can find different games. For many games, not only a paid version is available, but also a free one.
- Yahoo! Groups". This is the most popular social network that brings people together by interests. Those who wish to find a brother of interest can join existing groups or create a new one. The service is very convenient and informative. The only thing that can stop the search for colleagues in the interests of a Russian user is the lack of a Russian-language version of the web resource.
In general, it should be noted that the functions of the Russian-language Yahoo are too truncated. Among the variety of functions that are available on the international portal, Yahho! In Russian, it has only mail, instant messenger, news and a catalog of sites.
At the same time, English-speaking users can get answers to any questions, find a job, check the weather for the coming days, get acquainted with television programs, financial news, learn new opportunities in the Yahoo! Small Business (hosting, domain names, website builder and more).
Thus, although Yahoo! is one of the oldest international search engines, it is likely that due to the lack of a Russian-language version of the portal, in April 2009 this search engine was only in 13th place in Runet.
Features of promotion in Yahoo
Search promotion of a site in Yahoo is different from promotion in Google or Bing. In order for the site to rank better in Yahoo, you need to add it to the search engine directory. The addition is paid and is paid every year. There are also nuances when working on links and diluting texts with links for the Yahoo search engine.
But in practice it is shown that if the site is in the top in Google, then in Yahoo it will occupy the top ten. Those who actively use this search engine get significant advantages over other SEOs. Yahoo has the following features:
- Permanent trust in sites. If once a site has become a leader in the issuance of requests, then it will remain at the top all the time. If once you managed to promote the site, then you are guaranteed to stay in the TOP for a long time. Sometimes even the wrong actions of web resource owners do not lower the rating of the site. For old people, Yahoo is a favorable factor, but what about newbies? It is necessary to constantly fill the sites with new content, thereby gaining the trust of the search engine. If you succeed, then you can promote an old man with uninteresting or irrelevant content.
- Keywords. Everyone knows that search engine optimization requires website owners to include keywords and keywords in content. Yahoo is very demanding on key phrases, and, accordingly, you can increase the chance of getting into the TOP if you correctly create keywords and their number.
- Links. Yahoo doesn't require you to provide links to a topic. The main thing for Yahoo is not the quality of links, but their quantity. Therefore, you can advertise your site anywhere, indicating links to it.
- Long indexing. A special difference between Yahoo is the long indexing of sites. This process takes a very long time, as the system completely checks your site. Therefore, it is necessary to update the materials of old pages as often as possible. This will give the search engine a clear reason to boost your site to the top.
Since website promotion only in Yahoo for the Russian-speaking space is unacceptable, you need to choose the golden mean. It is not necessary to indicate links only of a non-thematic nature with the expectation of Yahoo, forgetting about Yandex or Google.
Due to such actions, such search engines may block your site from being viewed for their queries. Therefore, you need to pay special attention to the quality of the links created, as well as the thematic focus. This will allow your site to get high positions in Yahoo, but will not let you lose these positions in other search engines.
The Yahoo search engine is a large search engine that holds a decent share of the search market and provides good traffic to the promoted web resources, and is also considered the leader of the search market in Japan.
When promoting a site in the English-speaking segment and other regions, you need to remember that Yahoo gives potential visitors.
DuckDuckGo servers are located in the United States. The peculiarity of this search engine is that it has an open source code. The privacy and confidentiality of all incoming information is guaranteed: nothing about the user is reflected in the database, there is no search history, cookies are used to a minimum.
Thus, the advantage of DuckDuckGo is that it generates objective, not personalized results, does not show themed advertisements. In addition, DuckDuckGo will be relevant if you need to find something on foreign Internet resources. The fact is that Google and Yandex are focused on Runet, even if the keyword is not in Russian.
Features of the search engine DuckDuckGo
This system has very interesting features, and does not just copy the capabilities of the leaders. Let's take a look at these features. The official duckduckgo.com website notes the following benefits:
- Improved search mechanism (including images, videos and places) while maintaining privacy.
- The presence of direct answers to your queries in the search results.
- Appearance customization.
- Possibility of direct search in other services (Facebook, Amazon, Wikipedia).
- Easy to use and install DuckDuckGo.
- History of DuckDuckGo
The main feature of the search engine is to preserve the privacy of user data. That is, the service does not monitor and does not remember your activity, history and other data. This principle made it possible to attract a certain audience, and after Snowden's speech and the adoption of a law in America to collect information about users, it greatly increased it.
According to Wikipedia, after such statements, the number of users of the system doubled and amounted to 3 million requests per day. On the DuckDuckGo website, in the About project category, the most interesting facts about the history of the service since its opening in 2008 are presented.
For example, in 2011 the company placed a billboard "Google tracks you. We don't" in the backyard. At this time, the first official employee appeared, and a move was made from the basement to a spacious office.
In February 2012, 1 million requests per day were received, after which the figure skyrocketed - 2.3.4.5 million and, finally, 10 million requests per day in the summer of 2015. The DuckDuckGo search engine has become available in Safari and Firefox.
Lack of filters and search results
The creators of the project believe that one of the main disadvantages of modern search engines is to provide users with results that, in the opinion of these systems, suit users best.
To do this, you need to take into account the various preferences of the user, his browsing history and a whole list of other factors. Why should some other algorithm decide what exactly you want to find? The DuckDuckGo search engine allows you to get rid of this limitation and simply get an answer to any of your queries.
We do not know what the service was at the very beginning of its existence, but today users have the opportunity to select a region to take into account the location. Georeferencing is voluntary.
As for the results in the issue, the picture is a bit specific:
- The TOP on demand restaurant with the included region Ukraine has almost half of the links to restaurants in Moscow and St. Petersburg. And even with the included Ukrainian language for the search, which only affects the quote from Wikipedia.
- Confused mark in the lower right corner "in collaboration with Bing." Such a response can be seen on the request "wordpress blog" - it was indicated "in cooperation with Yandex". It turns out that results from other search engines are used? English-language queries for different regions gave more attractive results. Probably, the service does not fully function with Runet.
In general, personalization of issuance is a controversial issue. The position of the creators of the search engine DuckDuckGo can be understood, it makes sense. But if you specify the same request for a restaurant in Google, then it immediately shows a map and places nearby.
Comfortable or not is a rhetorical question. If a person is looking for an institution nearby or in his city, then this makes the job much easier. On the other hand, to clarify the location, you can always add a phrase to the query.
Instant replies
Instant Answers, or Instant Answers, is another interesting feature of this search engine. The site duckduckgo.com says that the main goal of the service in the long run is to provide the user with specific answers to his requests, and not a set of links.
Therefore, special sources may apply for some issuance groups. For example, in the English-speaking segment of the network, it is more convenient to search for restaurants through Yelp, music - in SoundCloud, weather - on forecast.io. Something like the Knowledge Graph from Goggle, which displays the maximum data on the request page. But for the most part, it uses its own power, while DuckDuckGo's search relies on third-party services.
In the list of instant responses, you can find unusual options, for example, FedEx parcel tracking, base64 decoding function, the ability to count the number of characters in a line, identifying the IP address and file extension. In short, the duckduckgo com service is a so-so tool for getting answers for all occasions.
Search on third party sites
The DuckDuckGo search engine allows you to directly find data in many other sites: Amazon, Wikipedia. After entering the passphrase, you will be redirected to the appropriate service. Today, more than 7,700 different requests are being made. They can be found here.
For the convenience of users, they are sorted into sections: entertainment, news, multimedia, services, shopping, technology, translations. There is a search form with which you can instantly find a suitable query.
By the way, Yandex provides 16 different functions (from maps to translations). All queries and formats begin with an exclamation point. The most popular ones are marked in tips on duckduckgo.com. It is enough to enter a request with an exclamation point! and you will see the suggested options.
In general, this service is very interesting. There are original functions of the system, namely search on other sites and instant answers. Customizing the look of DuckDuckGo in your browser is also a cool idea. With the help of them and data privacy, the service can compete for an even greater percentage of Internet users.
Reviews were found on the network that the project is somewhat similar to the young Google, which focused only on its 10 blue lines in the issue. This is all good, but what awaits in the future - no one knows. Somewhere in the review it was written that the DuckDuckGo search engine does not place ads, but now it is.
You can turn off these ads, but all changes are inevitable. It's one thing when it works out in a company of 5-10 employees, when the volumes increase, more and more calls and requests appear. If the system can keep its main features, then the audience will appreciate it.
Not Evil is based on the anonymous Tor network. In order to use this search engine, you need a special browser, for example, Tor.
With Not Evil, you can get into places where law-abiding citizens are forbidden to enter. Unfortunately, the authorities continue to block Internet projects for no reason, and they are not going to stop. Not Evil is the key to all doors.
Not Evil is a system that searches the anonymous Tor network. To use it, you need to go to this network, for example, by launching a special browser with the same name. Not Evil is not the only search engine of its kind.
There is Look (default search in the Tor browser, accessible from the regular Internet) or TORCH (one of the oldest search engines on the Tor network) and others. Today we will focus on Not Evil because of an interesting allusion to Google itself (just look at the start page). This system searches where Google, Yandex and other search engines are denied access in principle.
The Tor network has a lot of resources that are unrealistic to find on the law-abiding Internet. And as the authorities tighten control over the content of the Web, their number will increase. Tor is a kind of network within a network: with its own social networks, media, torrent trackers, marketplaces, blogs, libraries, and the like.
Using this search engine, you can save a lot of your time and maintain complete confidentiality. Very simple and intuitive interface. Previously, this project was called TorSearch, and it had a pretty impressive audience. Search results are good, but it is still very far from ideal.
YaCy
YaCy functions in the same way as P2P networks. Any device (computer, tablet or smartphone) on which this search engine is installed independently searches for information. In other words, it turns into a search robot.
Access to the results is provided to everyone who has downloaded YaCa. Among these results may be data from the depths of the Internet and non-indexed pages.
It is impossible to say whether this search engine is better or worse than others, since it is built differently. But due to the lack of a common server, two things are obvious: the issuance is not affected by previous actions and censorship is impossible.
YaCy is a decentralized search engine that operates on the principle of P2P networks. Each computer on which the main software module is involved scans the Internet on its own, that is, it is an analogue of a search robot. The results obtained are collected in a single database, which is used by all YaCy participants.
Why is this service better than other systems?
This is a completely different approach to searching. The lack of a shared server and owner company makes the results independent of anyone's preferences. If you are a supporter of open source software and a free internet that is not subject to the influence of government agencies and large corporations, then YaCy is a good choice.
Also, thanks to him, you can search inside a corporate or other autonomous network. Although not very useful in using YaCy, it is considered a worthy alternative to Google in terms of the search process.
Key features and advantages of YaCy:
- Decentralized search and storage of the index - in the DHT network.
- Work in three modes: global search index, group index and local index.
- No ads.
- The absence of any censorship.
- Wide range of settings.
Main cons:
- Search time (due to the decentralized nature, searching for unclaimed content can take a long time.
- Relevance (the relevancy algorithm has been redesigned in these three years, but it still falls short of the search giants).
What has changed in 3 years?
- Support for Russian-language search queries has been introduced.
- There were hints at the time of entering the search query.
- Lookahead search - while you are typing, the service is already looking for your content.
- Search and search algorithms have become better - a more relevant result and millions of other small changes under the hood.
How does YaCy work?
Indexing can be done in the following ways:
- forced indication of the start of scanning;
- remote request from another peer;
- setting up proxying of all requests from the browser to YaCy with further indexing of what you read.
After the database is compiled, it is stored locally, sending out to the network the data that you have it, and distributed by DHT to neighboring peers. During the search, YaCy first looks for results in the local index, and then in the global one, contacting peers via DHT that have data about the index of a particular query.
Pipl
Pipl was created in order to search for a specific person. The authors of this search engine claim that it is more efficient than Google and Yandex, as it relies on pages on social networks, lists and all kinds of databases.
Pipl has one drawback - it is bad at looking for people in Russia. But if you need to find a relative or friend from the US, then please.
The wider the World Wide Web, the more often it is used to find people and information about them. Now we are not talking about a dating site, but about situations where you need to find contacts or information about a particular person. If we talk about social networks, then there you can find only data about registered users.
To solve global problems, it is necessary to use special search engines. The most advanced free people search service on the web is Pipl.
This system allows you to find people by last name, first name, nickname, email address, phone number, you can additionally mark the location. The request is entered in various languages. Search results include links to personal profiles on Google Profiles, Facebook, professional profiles on LinkedIn, microblogs on Twitter, and other resources.
If the name is indicated in English, then in the second stage of the search it is possible to narrow its circle using filters, marking the country, region, age of the person. A feature of this service is the appeal to sources of the deep Internet, access to which is limited, and the information available on them is not indexed by simple search robots.
But the volume is very large, the hidden Internet correlates with the surface ones like the underwater part of an iceberg with the visible one, only there is hundreds of times more data there.
The source of information is not only profiles in social networks, analysis of closed databases, resumes, questionnaires that users of various resources fill out - online trading platforms, online auctions like Amazon, messaging systems like ICQ.
Collecting piece by piece information about different people, the Pipl system creates its own dossier on each, more detailed than on other resources. A unique identification mechanism is used, with the help of which you can find the right person.
Registered users of Pipl agree to the processing of their personal data and adding them to the database of the system, which is constantly expanding. Users cannot be persons under 13 years of age. The list of links leads from Pipl to partner sites.
One of them, spokeo.com, allows you to access hidden profiles on social networks, on the other, peoplefinders.com, you can find out the addresses and phone numbers of search targets for a fee, and the InstantCheckmate.com resource has data on arrests and convictions of various people.
Find Sounds
Like Pipl, it is a specialized search engine. But unlike him, he is looking not for people, but for sounds - nature, cars, etc. in sources open to all. This search engine does not process requests in Russian, but you can solve the problem through tags, which are in abundance.
In FindSounds there are no photos, no videos, nothing else but sounds. In the settings, you can check the box next to one of the formats and quality of the audio track. All the sounds that will be in the issue can be downloaded to your computer, which is very convenient. It is possible to download a sample and already look for something similar on it.
To get an idea of the kind of sounds FindSounds searches for, you can check out an impressive list of examples. The service supports the great and mighty Russian language, so you can explore a far from complete list of possible sounds and effects on the page with examples.
The search result in FindSounds looks unusual. In addition to the link to the file, there is its frequency response, a mini-lane (to listen to the sound or effect at the moment) and even the buttons "email this sound" or "tweet this sound". When searching in FindSounds, you can use the following settings:
- file format - AIFF, AU, MP3, WAVE;
- number of channels - mono, stereo;
- sampling depth (bit depth) - 8-bit, 16-bit;
- sampling rate - 8000 HZ - 44100 HZ;
- maximum file size - from 16K to 2Mb.
FindSounds is a highly specialized tool, so it is useful for musicians, online beat developers or professional DJs. It will also appeal to ordinary users who want to diversify the operating system environment with sound effects. This service has an impressive index of sounds of a very different nature, which makes it unique in conditions of low competition.
FindSounds has a list of unique add-ons:
- Waveform Display Results - This is where the results appear in the form of a colorful image. By the shape of the graph on it, you can judge the content of the found sound file. Usually this is a graph of loudness versus time. The color of the graph also changes, it all depends on the frequency group. This is very useful for sound engineers and other users who deal with sound professionally. A special FindSounds Palette program comes to help in reading graphics.
- Find Similar Results - allows you to expand your search by showing similar results. It is used as a last resort, as the level of relevance is low.
- Show Page - opens the page where the searched sound fragment was found.
But there is a warning to keep in mind when using this unusual search engine:
“When you search for sound effects when you use FindSounds.com, you get links to sound bites found on websites around the world. The sound effects featured in found sound bites are subject to national and international copyright laws. We do not give permission to use them."
In simple words, you just need to write to the owner of this file and ask for permission to use it.
WolframAlpha is a real treasure trove for those whose educational or professional activities are related to computing. Instead of the standard issue (links to sites), this search engine provides a ready-made answer to the question. Try typing "compare population in Moscow and St. Petersburg". You will see a large number of charts, graphs and tables with detailed explanations.
With WolframAlpha, you don't have to spend time analyzing and organizing information. The search robot will do everything itself and show only the result.
WolframAlpha Features
You can use other search tools besides Google. For example, WolframAlpha is a popular intelligent search platform that can also search the web. But this service is different in that it can process requests in natural language and understand them, as a result of its work, give an exact answer to the question or request.
WolframAlpha is a one-stop reference where you can find just about anything with an effort. Unlike Google or other standard search engines, the WolframAlpha search results provide a ready-made answer to a question, rather than pages where you still need to find information of interest.
Therefore, this service saves a lot of time and effort and can be used very actively in fact checking. Here is a list of possible use cases for WolframAlpha:
- The service helps to find detailed information about famous people and events. You can find anything on the Internet, but WolframAlpha will make this list simple and comfortable. In particular, this concerns the search for information about famous people. WolframAlpha algorithms are able to piece together and provide in a convenient format almost everything that is on the network. It is important to correctly put the request. For example, using Wolfram Alpha, you can find out which films starred two actors at once, or which video games were released in a particular month. The service will provide a list of Nobel laureates of a certain year, indicating their age, place of birth, field of science. Wolfram Alpha will find the first phrase of a classic novel or bible.
- The service works fine with dates. A query in the form of a date in Wolfram Alpha produces an interesting result. For example, if you specify a date as a query, the search engine will calculate how many days, weeks, months and years ago it was or will be. For dates from the past, the service displays important events that happened on that day - the beginning of wars, famous plane crashes and natural disasters, historical events, or the birthdays of famous people. If you set a date interval (within reasonable limits), the service will display the most important events that occurred in a certain period. Do not count on Wolfram Alpha to correctly process a span of a decade, but six months or a year is a completely reasonable period for which the service can work as the user needs.
- The weather on a particular day. Working with dates in Wolfram Alpha is done in such a diverse way that, as part of our review, we will not be able to tell in detail about all the features of this service. And this means that to check the facts - quickly and comfortably, without long googling and sifting web resources, by the efforts of webmasters - Wolfram Alpha will instantly give the correct and visual result. For example, the service makes it easy to find out what the weather was like on a certain day in a designated city. Detailed geographical information about the city will immediately be displayed, supplemented by a map with weather dynamics in the desired city over the past 50 years. Moreover, you can specify a climate query - Wolfram Alpha answers this question perfectly.
- Comparisons using Wolfram Alpha. The service has a very comfortable comparison tool. It is enough just to write two concepts separated by a comma or supplement them with a qualifying word, and Wolfram Alpha will make a comparative analysis and show a convenient table. For example, you can compare the population of two cities, two bicycles, an airplane. Thus, it is possible to compare the attendance of two web resources and other information about them, about two magazines or programs.
- Image identification with Wolfram Alpha. The service has its own image identification project. Unlike reverse lookup services, this service cannot recognize who is in the picture. Its task is to answer the question of what exactly is shown in the picture. For example, he can recognize a car, truck, plane, household appliances, and so on. Wolfram Alpha developers note that such a smart image search is a more difficult task than answering the most complex text queries, and promise to modernize this service in the future.
- Identification of parameters for photography. Many modern digital cameras can calculate depth of field and aperture, but you can turn to Wolfram Alpha for help with this task. You need to ask a question about the depth of field, and the service will calculate the aperture value and focal length. Wolfram Alpha can calculate the main aperture settings, hyperfocal distance, and a number of other parameters.
As the inventors assure, Dogpile has a special ranking algorithm, it collects and gives out the best of the best. It is indispensable if you cannot find information either in Google or Yandex. And it happens, believe me, especially when you are trying to learn something serious, to get to the bottom of the truth.
A nice bonus Dogpile, which can not be said - a minimum of advertising.
BoardReader allows you to search for information on social networks, forums, Q&A services, and other sites with predominantly textual content.
The BoardReader has several filters. You can search for a post and its comments based on criteria such as language, site name, and publication date.
This search engine is useful for PR specialists and all other mass media specialists who study the opinion of the population on a particular topic.
Finally
Alternative search engines do not have a good fate. Some of them die very quickly, having not lived even a couple of years, others, although they remain, serve less than one percent of the target audience of Google and Yandex and, accordingly, do not have solid financial prospects.
If you go through the examples in the article again, it will become clear that alternative search engines are either concentrating in a narrow area that the giants Google and Yandex have not yet paid attention to, or they are testing some kind of ranking algorithm.
If more people need to search the anonymous Tor network than now, then Google and Yandex will definitely offer something similar. This also applies to other chips of alternative search engines.
Be that as it may, but alternative search engines provide what Google and Yandex are not capable of. And that's great.
In the early years of the development of the Internet community, active Internet users were a minority, and the amount of information on Internet resources was relatively small. For the most part, only employees of scientific laboratories and large educational institutions had access to the world information network. In general, the use of a network resource was not as relevant as it is today.
History of search engines
A big step towards the spread of the Internet to the masses was the appearance in 1990 of the website info.centr.ch. This site was the first public directory of Internet sites. The creator is British scientist Tim Berners-Lee, who is also considered the creator of URI, HTTP, World Wide Web and URL. Since that moment, Internet sites have become relevant not only in specialized circles of users, but also among ordinary owners of home computers. In this directory, for convenience, information resources have been arranged on the basis of groups on similar topics, which greatly facilitates the search for information.
But progress did not stop there, and in 1994, the search technology developed by Carnegie University, known as Lucos, was born. This catalog, which was created by Michael Maldin, started with a resource of more than 50,000 documents. In Lucos, queries were considered approximate matches of the query, and the search result was ranked depending on the match between the input and output information. And also, there was a constant replenishment of the resource with new Internet pages. By November, Lucos had over 55 million pages and documents, far more than any document catalog of the day.
At the end of 1994, the Infosek resource appeared. It had a number of advantages over other resources. For example, adding sites by the user to the catalog database in real time.
The new search engine monster in 1995 was AltaVista. She quickly earned popularity among Internet users and took a leading position in her field. Its main feature was the ability to formulate queries in a natural, colloquial language, as well as users were allowed to add their own URL addresses. But still, the main merit of AltaVista was the support of multiple language packs, such as Korean, Japanese, and Chinese, as well as Russian.
A huge step in search technology was the emergence on the Internet of a new search engine, the name of which is now heard by every user, namely Google. In 1997, L. Page and S. Brin from Stanford University introduced new features into the search algorithms of their offspring. In the search, systems of relevance of the search results issued by the system were used, and morphology and possible spelling errors were taken into account when querying.
There are three major leaders in the search engine market these days: Bing, Google, and Yahoo. They have search algorithms and databases of their own production at their disposal. Many search engines, of which there are many, use the developments of these three titans among search engines.
Thanks to search engines, it has become easier for an ordinary person to discover the vast expanses of the information field. Without their development, it is impossible to improve the ways of exchanging information between people.
The architecture of a search engine typically includes:
Encyclopedic YouTube
1 / 5
✪ Lesson 3: How a search engine works. Introduction to SEO
✪ Search engine from within
✪ Shodan - Black Google
✪ Cheburashka search engine will replace Google and Yandex in Russia
✪ Lesson 1 - How a search engine works
Subtitles
History
| Chronology | ||
|---|---|---|
| Year | System | Event |
| 1993 | W3Catalog ?! | launch |
| Aliweb | launch | |
| JumpStation | launch | |
| 1994 | web crawler | launch |
| infoseek | launch | |
| Lycos | launch | |
| 1995 | AltaVista | launch |
| Daum | Base | |
| open text web index | launch | |
| Magellan | launch | |
| Excite | launch | |
| SAPO | launch | |
| Yahoo! | launch | |
| 1996 | Dogpile | launch |
| Inktomi | Base | |
| Rambler | Base | |
| hotbot | Base | |
| Ask Jeeves | Base | |
| 1997 | Northern Light | launch |
| Yandex | launch | |
| 1998 | launch | |
| 1999 | AlltheWeb | launch |
| GenieKnows | Base | |
| Naver | launch | |
| Teoma | Base | |
| Vivisimo | Base | |
| 2000 | Baidu | Base |
| Exalead | Base | |
| 2003 | info.com | launch |
| 2004 | Yahoo! Search | Final launch |
| A9.com | launch | |
| sogou | launch | |
| 2005 | MSN Search | Final launch |
| Ask.com | launch | |
| Nigma | launch | |
| goodsearch | launch | |
| Search Me | Base | |
| 2006 | wikiseek | Base |
| Quaero | Base | |
| Live Search | launch | |
| ChaCha | run ( beta) | |
| Guruji.com | run ( beta) | |
| 2007 | wikiseek | launch |
| Sproose | launch | |
| Wikia Search | launch | |
| blackle.com | launch | |
| 2008 | DuckDuckGo | launch |
| Tooby | launch | |
| Picollator | launch | |
| Viewzi | launch | |
| Cuil | launch | |
| Boogami | launch | |
| LeapFish | run ( beta) | |
| Forestle | launch | |
| VADLO | launch | |
| powerset | launch | |
| 2009 | bing | launch |
| KAZ.KZ | launch | |
| Yebol | run ( beta) | |
| Mugurdy | closure | |
| Scout | launch | |
| 2010 | Cuil | closure |
| Blekko | run ( beta) | |
| Viewzi | closure | |
| 2012 | WAZZUB | launch |
| 2014 | Satellite | run ( beta) |
At an early stage in the development of the Internet Tim Berners-Lee maintained a list of web servers hosted on the site CERN. There were more and more sites, and manually maintaining such a list became more and more difficult. On the site NCSA there was a special section "What's new!" ( English What's New!), where they published links to new sites.
The first computer program to search in Internet was a program Archie (English archie - archive without the letter "c"). It was created in 1990 by Alan Emtage, Bill Heelan, and J. Peter Deutsch, computer science students at McGill University in Monreale. The program downloaded lists of all files from all available anonymous FTP servers and built a database that could be searched by filenames. However, Archie's program did not index the contents of these files, as the amount of data was so small that everything could be easily found by hand.
Development and distribution of the network protocol gopher, invented in 1991 by Mark McCahill in University of Minnesota, led to the creation of two new search programs, Veronica and Jughead. Like Archie, they looked up filenames and headers stored in Gopher's index systems. Veronica ( English Very Easy Rodent-Oriented Net-wide Index to Computerized Archives) allowed keyword searches for most Gopher menu headings across all Gopher lists. Jughead program ( English Jonzy's Universal Gopher Hierarchy Excavation And Display) retrieved menu information from certain Gopher servers. Although the name of Archie's search engine was not related to the comic book series "Archie", nevertheless Veronica and Jughead are characters in these comics.
By the summer of 1993, there was not yet a single system for searching the web, although numerous specialized directories were manually maintained. Oscar Nierstrass in Geneva University wrote a number of scripts for Perl who periodically copied these pages and rewrote them into a standard format. This became the basis for W3Catalog ?! , the first primitive web search engine, launched on September 2, 1993.
Probably the first crawler written in Perl was "World Wide Web Wanderer"- Matthew Gray's bot from June 1993. This robot created a search index " Wandex". The purpose of the Wanderer robot was to measure the size of the World Wide Web and find all web pages containing the words from the query. In 1993, a second search engine " Aliweb". Aliweb did not use search robot, but instead waited for notifications from website administrators about the presence of an index file in a certain format on their sites.
JumpStation, created in December 1993 by Jonathan Fletcher, searched and indexed web pages using a crawler, and used web form as an interface for formulating search queries. It was the first Internet search tool that combined the three essential functions of a search engine (verification, indexing, and actual search). Due to the limited computer resources of the time, indexing, and therefore search, was limited to only the titles and titles of the web pages found by the crawler.
Search engines have participated in "Dot-com Bubble" late 1990s. Several companies entered the market in spectacular fashion, earning record profits during their initial public offer. Some have abandoned the public search engine market and only work with the corporate sector, such as Northern Light.
Google took up the idea of selling keywords in 1998, when it was a small company running a search engine at goto.com. The move marked a shift for search engines from competing with each other to becoming one of the most profitable business ventures on the Internet. Search engines began to sell the first places in search results to individual companies.
The Google search engine has been in a prominent position since the early 2000s. The company achieved a high position due to good search results with the help of the algorithm PageRank. The algorithm was presented to the public in the article "The Anatomy of Search Engine" written by Sergey Brin and Larry Page, founders of Google. This iterative algorithm Ranks web pages based on an estimate of the number of hyperlinks to a web page, assuming that "good" and "important" pages get more links than others. Google's interface is designed in a spartan style, where there is nothing superfluous, unlike many of its competitors who built the search engine into the web portal. The Google search engine has become so popular that imitators of it have appeared, for example, Mystery Seeker(secret search engine).
Search for information in Russian
In 1996, a search was implemented taking into account Russian morphology on the Altavista search engine and the original Russian search engines were launched. Rambler And Aport. On September 23, 1997, a search engine was opened Yandex. May 22, 2014 by the company Rostelecom a national search engine was launched Satellite, which at the time of 2015 is in beta testing. On April 22, 2015, a new Sputnik service was launched. Children specially for children with increased safety.
Methods gained popularity cluster analysis and search by metadata. Of the international machines of this kind, the most famous was "Clusty" companies Vivisimo. In 2005 in Russia with the support of Moscow State University search engine launched Nigma”, which supports automatic clustering. In 2006, the Russian metamachine was opened Quintura, which offers visual clustering in the form clouds tags. Nigma also experimented with visual clustering.
How the search engine works
The main components of the search engine: search robot , indexer , search engine.
As a rule, systems work in stages. First, the crawler gets the content, then the indexer generates a searchable index, and finally, the crawler provides the functionality to search the indexed data. To update the search engine, this indexing cycle is repeated.
Search engines work by storing information about the many web pages they get from HTML pages. Search robot or "crawler" ( English Crawler is a program that automatically traverses all links found on a page and highlights them. The crawler, based on links or based on a predefined list of addresses, searches for new documents that are not yet known to the search engine. The site owner can exclude certain pages using robots.txt, using which you can disable the indexing of files, pages or directories of the site.
The search engine analyzes the content of each page for further indexing. Words can be extracted from titles, page text or special fields - meta tags. An indexer is a module that analyzes a page, after splitting it into parts, using its own lexical and morphological algorithms. All elements of a web page are isolated and analyzed separately. Web page data is stored in an index database for use in subsequent requests. The index allows you to quickly find information on the user's request. A number of search engines, like Google, store the original page in whole or in part, the so-called cache, as well as various information about the web page. Other systems, like AltaVista, store every word of every page found. Using the cache helps speed up the extraction of information from already visited pages. Cached pages always contain the text that the user specified in the search query. This can be useful when the web page has been updated, that is, it no longer contains the text of the user's request, and the page in the cache is still old. This situation is related to the loss of links ( English linkrot) and user-friendly ( usability) Google's approach. This involves issuing short chunks of text from the cache containing the query text. Active principle of least surprise, the user usually expects to see the search words in the texts of the received pages ( user expectations). In addition to speeding up searches using cached pages, cached pages may contain information that is no longer available elsewhere.
The search engine works with output files received from the indexer. The search engine accepts user requests, processes them using an index, and returns search results.
When a user enters a query into a search engine (usually via keywords), the system checks its index and returns a list of the most relevant web pages (sorted by some criterion), usually with a short annotation containing the title of the document and sometimes parts of the text. The search index is built according to a special technique based on information extracted from web pages. Since 2007, the Google search engine allows you to search based on time, create the documents you are looking for (call the "Search Tools" menu and specify the time range). Most search engines support using in queries boolean operators AND, OR, NOT, which allows you to refine or expand the list of searched keywords. In this case, the system will search for words or phrases exactly as entered. Some search engines allow approximate search, in this case, users expand the search area by specifying the distance to keywords . There are also conceptual search, which uses a statistical analysis of the use of the search words and phrases in the texts of web pages. These systems allow you to compose queries in natural language. An example of such a search engine is the site ask com.
The usefulness of a search engine depends on relevance the pages she found. While millions of web pages may include a word or phrase, some may be more relevant, popular, or authoritative than others. Most search engines use ranking methods to bring the "best" results to the top of the list. Search engines decide which pages are more relevant and in what order results should be shown in different ways. Search methods, like the Internet itself, change over time. Thus, two main types of search engines appeared: systems of predefined and hierarchically ordered keywords and systems in which inverted index based on text analysis.
Most search engines are commercial enterprises that profit from advertising, in some search engines you can buy for a fee the first places in the search results for given keywords. Those search engines that do not take money for the order of results, earn on contextual advertising, while advertising messages correspond to the user's request. Such ads are displayed on the page with a list of search results, and search engines earn every time a user clicks on advertising messages.
Search Engine Types
There are four types of search engines: robotic, human-driven, hybrid, and meta-systems.
- systems using search robots
- human controlled systems(resource directories)
- hybrid systems
- meta-systems
Search engine market
Google is the most popular search engine in the world with a market share of 68.69%. Bing occupies the second position, its share is 12.26%.
The most popular search engines in the world:
| Search system | Market share in July 2014 | Market share in October 2014 | Market share in September 2015 |
|---|---|---|---|
| 68,69 % | 58,01 % | 69,24% | |
| Baidu | 17,17 % | 29,06 % | 6,48% |
| bing | 6,22 % | 8,01 % | 12,26% |
| Yahoo! | 6,74 % | 4,01 % | 9,19% |
| AOL | 0,13 % | 0,21 % | 1,11% |
| Excite | 0,22 % | 0,00 % | 0,00 % |
| Ask | 0,13 % | 0,10 % | 0,24% |
Asia
In East Asian countries and in Russia, Google is not the most popular search engine. In China, for example, more popular search engine Soso ?! .
In South Korea, a search portal of its own design Naver used by about 70% of the population Yahoo! Japan and Yahoo! Taiwan are the most popular search engines in Japan and Taiwan, respectively.
Russia and Russian-language search engines
According to LiveInternet data in June 2015 on the coverage of Russian-language search queries:
- All-lingual:
- English speaking and international:
- AskJeeves(mechanism Teoma)
- Russian speakers- most "Russian-language" search engines index and search for texts in many languages - Ukrainian , Belarusian , English , Tatar and others. They differ from "all-lingual" systems that index all the documents in a row, by the fact that they mainly index resources located in domain zones, where the Russian language dominates, or otherwise limit their robots to Russian-language sites.
Some of the search engines use external search algorithms.
Quantitative Google Search Engine Data
The number of Internet users and search engines and user requirements for these systems is constantly growing. To increase the speed of searching for the necessary information, large search engines contain a large number of servers. Servers are usually grouped into server centers (data centers). Popular search engines have server centers scattered all over the world.
In October 2012, Google launched the Where the Internet Lives project, where users are given the opportunity to get acquainted with the company's data centers.
The Google search engine knows the following about the work of data centers:
- The total capacity of all Google data centers, as of 2011, was estimated at 220 MW.
- When Google planned to open a new 6.5 million m² three-building complex in Oregon in 2008, Harper's Magazine estimated that such a large complex would consume over 100 MW of electricity, which is comparable to the energy consumption of a city of 300,000 people. human.
- The estimated number of Google servers in 2012 is 1,000,000.
- Google's spending on data centers was $1.9 billion in 2006 and $2.4 billion in 2007.
The size of the World Wide Web indexed by Google as of December 2014 is approximately 4.36 billion pages.
Search engines that take into account religious prohibitions
The global spread of the Internet and the increasing popularity of electronic devices in Arabic And Muslim world, in particular in countries Middle East And Indian subcontinent, contributed to the development of local search engines that take into account Islamic traditions. Such search engines contain special filters that help users avoid accessing prohibited sites, such as sites with pornography, and allow them to use only those sites whose content is not contrary to the Islamic faith. Shortly before the Muslim month Ramadan, in July 2013, the world was introduced Halal googling- a system that gives users only halal"correct" links by filtering search results received from other search engines such as Google And bing. Two years earlier, in September 2011, the I'mHalal search engine was launched to serve users in the Middle East. However, this search service had to be closed shortly, according to the owner, due to lack of funding.
The lack of investment and the slow pace of technology diffusion in the Muslim world has hindered progress and hindered the success of a serious Islamic search engine. The failure of huge investments in Muslim lifestyle web projects, one of which was Muxlim. He has received millions of dollars from investors such as Rite Internet Ventures and now - according to the latest post from I'mHalal before it shut down - comes up with the dubious idea that "the next Facebook or Google may only appear in countries Middle East if you support our brilliant youth." Nevertheless, Islamic Internet experts have been busy for years determining what is right or wrong. sharia, and classify websites as " halal" or " haram". All former and present Islamic search engines are simply a specially indexed set of data, or they are major search engines such as Google, Yahoo and Bing, with a specific filtering system used to prevent users from accessing haraam sites such as nudity sites, LGBT, gambling and any other subject matter which is considered anti-Islamic.
Among other religion-oriented search engines, Jewogle, the Jewish version of Google, and SeekFind.org, a Christian site that includes filters to keep users away from content that could undermine or weaken their faith, are common.
Personal results and filter bubbles
Many search engines, such as Google and Bing, use algorithms to selectively guess what information a user would like to see based on their past activities on the system. As a result, websites only show information that is consistent with the user's past interests. This effect is called "filter bubble".
All this leads to the fact that users receive much less information that contradicts their point of view and become intellectually isolated in their own "information bubble". Thus, the "bubble effect" can have negative consequences for the formation of civic opinion.
Search engine bias
Although search engines are programmed to rank websites based on some combination of their popularity and relevancy, the reality is that experimental research indicates that various political, economic, and social factors influence SERPs.
This bias can be a direct result of economic and commercial processes: companies that advertise on a search engine may become more popular in organic search results on that search engine. The removal of search results that do not comply with local laws is an example of the influence of political processes. For example, Google will not display some neo-Nazi websites in France and Germany, where Holocaust denial is illegal.
Bias can also be a consequence of social processes, as search engine algorithms are often designed to exclude unformatted points of view in favor of more "popular" results. The indexing algorithms of the major search engines prioritize American sites.
search bomb is one example of an attempt to manipulate search results for political, social, or commercial reasons.
see also
- Qwika
- Electronic library#Lists libraries and search systems
- Web developer toolbar
Notes
Literature
- Ashmanov I. S., Ivanov A. A. Website promotion in search engines. - M. : Williams, 2007. - 304 p. - ISBN 978-5-8459-1155-1.
- Baikov V.D. Internet. Search for information. Website promotion. - St. Petersburg. : BHV-Petersburg, 2000. - 288 p. - ISBN 5-8206-0095-9.
- Kolisnichenko D. N. Search engines and website promotion on the Internet. - M.: Dialectics, 2007. - 272 p. - ISBN 978-5-8459-1269-5.
- Lande D.V. Search for knowledge on the Internet. - M. : Dialectics, 2005. - 272 p. - ISBN 5-8459-0764-0.
- Lande D. V., Snarsky A. A., Bezsudnov I. V. Internet: Navigation in complex networks: models and algorithms. - M.: Librokom (Editorial URSS), 2009. - 264 p. - ISBN 978-5-397-00497-8.
- Chu H., Rosenthal M.
The popularity of search engines in Russia, Ukraine and the world. The history of the development of search engines. Rambler and Aport are extinct dinosaurs.
I have always argued and will continue to assert that a narrow specialist who is not interested in anything but purely practical details will not be able to achieve real heights in his profession. Such a person becomes not “narrow”, but “narrow-minded” :-) Therefore, let's look at the history of the development of search engines, as well as what market share each of the search engines currently occupies. The popularity of search engines in Russia and Ukraine is a very significant moment for promotion.
History of search engines . In the distant 90s, when the Internet was just beginning to come into life, search engines in the modern sense of the word did not exist. The search for the required resource took place through site directories, where the Internet projects available at that time were sorted by headings. Since there were few users on the network, and there were few sites in general, this way of structuring data was quite convenient and adequate.
The first full-fledged search engine was the WebCrawler project. The fundamental difference from its predecessors was that this search engine carried out the so-called. full text search. Those. searched not only locally in some directory, not only in the Keywords meta tag, but analyzed the full text of the web page, which made it possible to successfully find any phrases (and not just those that the webmaster indicated as keywords in the description of his site).
Personally, the name WebCrawler doesn’t mean anything to me - it’s too far in the past, in Russia at that time not everyone heard the word Internet :-) But Lycos, which appeared in 1996, is already on the lips of older webmasters, in any case, the name is many are familiar. Soon after Lycos, the best search engine of its time, AltaVista, was launched - you must have heard that name. Until about 2000, AltaVista remained the world's leading search engine.
In 1998, Google was founded. Its developers, Sergey Brin and Larry Page, revolutionized the quality of search by developing the PageRank algorithm, which allows you to take into account the "weight" of a page, depending on the number of links to it. This improved the quality of search so radically that Google began to rapidly gain an audience and within a few years became the world's leading search engine, which it still remains. I remember, in about 2000, I watched a dispute between two students in our group at the institute that Altavista sucks, and Google searches better :-)
Aport search engine . The development of Agama, which was released in 1996. A project that for several years remained the leading search engine on the Runet. I am writing this and I begin to feel very ancient myself :-) After all, once, as students, it was Aport that we used to search. There were no other alternatives, and they were not needed, because Aport did its job perfectly.
An excellent start, however, did not provide an equally excellent development in the future. The aport was such a tasty morsel that it changed owners several times, each time selling for more and more. That, apparently, is what killed him. “Effective management and marketing” came to the fore, and the technical support was tritely “scored”. After that, Runet users turned their backs on Aport - fortunately, the other two main search engines - Rambler and Yandex - were actively fighting for a place in the sun.
I don’t remember the exact dates, but at first Aport abandoned its own ranking algorithms, started using Yandex data, and a couple of years ago it completely ceased to exist as a search engine. Now an online store is located on the famous domain, and not of the best quality.
Eh ... but once Aport also had its own catalog, which webmasters sought to get into with almost the same force as Yandex now. By the way, the first design for Aport was developed by Artemy Lebedev ... almost simultaneously with the design for Yandex.
Search engine Rambler . Rambler was launched all the same in 1996 and remained one of the main players in the market for almost a decade. Unlike Aport, which “faded away” very, very quickly, Rambler rolled down to the current “homeless position” slowly and with resistance.
Almost until the very last moment, Rambler continued to work with its own search engine, which, by the way, gave out good quality. Just think - for many years Rambler was the second Runet search engine, and the third was not the already half-dead Aport, but the mighty Google!
However, Rambler's market share was shrinking and shrinking, and in 2011 it was bought out by Yandex and began to produce results identical to Yandex. Thus ended the story of Rambler, once the strongest Russian search engine.
Search engines Google and Yandex . At the moment, in fact, there are only two relevant search engines - Yandex and Google. All kinds of Lucos and Altavistas have long been forgotten and do not function as search services. Rambler was bought by Yandex and now shows the results of the latter. Aport has turned into an online store. A very small share of the market is held by Yahoo and Bing - search from Microsoft. In Russia, in addition to Yandex and Google, there is also Mail.ru, but it does not have its own search engine, but uses search results from Google, making only minor adjustments to them.
The popularity of search engines in Russia, Ukraine and in the world . At the moment, the search market in Russia is divided approximately as follows:
Yandex - 63%
Google - 26%
Mail.ru - 8%
Rambler - 1.5%
Bing - 0.6%
In Ukraine, the ratio is significantly different, Yandex is well behind its main competitor, but is slowly increasing its market share:
Google - 60%
Yandex - 29%
Mail.ru - 6%
In the global search, there is only one leader, absolute and unconditional - Google.
By the way, you and I can feel a well-deserved pride in our country - only in Russia and China have their own search systems been created. Nowhere else in the world did high-quality search engines appear. In European countries - France, Germany and others - Google is used by up to 97-99% of users, i.е. there is essentially no alternative.
There is also a practical aspect to this - a Russian webmaster is more likely to get visitors from the search. It often happens that Yandex “loves” the site, but Google does not, or vice versa. But in both cases, the site has customers. Our Western colleagues work on the "all or nothing" principle.
There is a second point - promotion in Google and Yandex differs to a certain extent, and the desire to get into the top of both leaders leaves its mark on the site promotion model.
My book is published in hard copy. If this textbook turned out to be useful for you, then you can thank me not only morally, but also quite tangibly.
For this you need to go to
Why you need a BIOS flashing
Internet browser Google Chrome - full information and free download Russian version
What to do if the laptop battery is not charging