Today I want to consider the issue of the technical component of our sites, and I offer everyone to look under the "hood" of my site or blog. Is everything in order really with your site, as it seems at first glance? Let's talk about the Xenu program, which will help you find broken, outgoing and other unsuitable links on your resource.

Xenu Program - We carry out a technical audit on the site

Greetings, I completely disappeared and stopped doing my blog. More and more go to work with commercial sites. I will use a small break and share the useful Xenu program for webmasters, which has already reversed me. I work with her for quite a long time and constantly applied in practice.

Using the Xenu program you can find all outgoing and broken links on your website.

Briefly tell one interesting case, which happened a few months ago. On one of the sites over which I work, in particular, the online store has taken place. My colleague, thank you very much, noticed in several cards with a commodity, suspicious links to other sites. These links can no longer be present on our resource, as we use it for customers. Immediately launched Xenu, there are quite a few pages on this site, so check takes about 1-2 hours.

At the end of the check, we found about 20 outgoing links, including 18+ sites and an online casino. It was also possible to calculate the attacker, as he left the outgoing link and on himself. Yes, there is such an exchange on the Internet, which sells links from hacked sites, perhaps someone heard about it.

As a result, we all cleaned, just found a couple of interesting scripts and replaced passwords. The consequences were not very tangible, but lost a little traffic in Google, as they responded about a week later ... Let's go to practice.

Check outgoing and external links

Based on the stories that I told you if you suspect that something is wrong with your resource (for example, the loss of positions or traffic), I advise you to periodically drive your website or blog using this program. You can even do it now, in order to prevent, with me.

The first thing we'll see is how to detect external links. Xenu You can download the program for checking external links.

1. Run the program, enter the address of our site and click OK.

We are waiting for the robot, runs on all our pages. Depending on how many pages you have, scanning lasts from a few minutes to several hours.

2. As soon as the robot bypassed all pages, we see a list of different links that are present on our resource. In order to be easier to see and find outgoing links from your site, we sort links over the field "Address". We look at the top of the list and end, and here are our external links.

3. In order to quickly find the pages from where the external link is made, it is necessary when you click on, right-click on the "bad" link to select "URL PROPERTIES".

We are looking for broken links on the site with Xenu

Now let's check if there are broken links on the site, which give 404 code and how to find them. The main reason for the appearance of such links is that the address (URL) of the page on which the link is logged by changed or the page is removed, and the link remained. There are also other reasons.

With such a sorting, you can also find and links for which redirect happens, which is also very useful.

I told you the main two points on finding external and broken links on the site using Xenu. The program is free and as far as I know, is already quite old and not updated, but for us the main thing that works.

I also use this program, with the "first acquaintance" with the site that is starting to do.

On this, everything, if you have another interesting software on this topic, share in the comments!

Good day! Last week, returned from the village where he spent an unforgettable time of full relax! The weather surrendered excellent, we spent the days in the rivers, lakes, forests, fields. Fresh air and nor soul around - this is a real holiday!

We continue to study the topic of blog optimization on WordPress. Earlier, we considered in detail the issue of checking the site on and manually and with the help of the plugin. Today we will focus in detail on 2 desktop programs that perform these tasks automatically, we will analyze their basic settings and technical audit parameters. It is Xenu's Link Sleuth and Netpeak Spider, free tools for SEO-analysis blog.

How to use the Xenu program.

It is not difficult to work with this program, and as a result, a detailed and thorough report of all links available on the site are obtained. In addition to its main destinations, checking broken links, Xenu can perform other tasks:

• drawing up a site map;
• definition of page loading time;
• search for unwitting titles;
• identifying pages with a significant level of nesting;
• determining the number of external and internal links on the site;
• detection of pictures without an alt attribute.

Indicated moments are very important for the development of the resource. The task data solution will serve to optimize your WordPress Blog. Therefore, I suggest you carefully examine the instructions for working with Xenu Link Sleuth. Let's start.

This application is free. You can download the Xenu program by reference: xenus-link-sleuth.en.softonic.com..

Unzip the file and run setup.exe. After installing the program, you can start setting up its basic parameters. It should be noted that Xenu is written for Windows and does not work with other operating systems.

An English-speaking application interface, but it is concise and simple enough. To create a new check, you need to click on the first icon in the upper left corner of the program window or go along the path "File" - "Check URL". As a result, the "Xenu's Starting Point" window opens with the basic settings:

At the top enter the address of the inspected resource. If you want to check the links to external resources in check, you should leave a tick near the item « Check.externallinks ». Button « Localfile » Designed to select a file on your computer if you want to analyze such.

In field « Consider.URLS.beginningwith.tHISaS 'internal ': " You can add such pages that are directly related to the site to be checked, but have a different address from the main page. For example, https: //forum.Vashsite.ru .

In the same field « Do.not.check.any.URLS.beginningwith.this: " You can list those links that you want to exclude from the test of the XENU program.

At the bottom there is a button « e-mail ". It will require you if you want to get a report on the blog analysis to your e-mail using SMTP-Server:

« e-mailreportand.quit.when.done » - select this item if you want to get the entire report immediately upon completion of the application;

« e-mailoNLY.if.tHEREarebroken.links » - When this option is selected, the report will come if broken links are found.

In addition, Xenu provides a large number of additional settings. They can be called by clicking on the button « More Options ».

The Basic Options tab opens. First parameter - « ParallelThreads ». It regulates the number of parallel synchronous requests. In other words, what the number of links will be checked at the same time. It is recommended not to exceed the default value so as not to overload the server and not affect the correctness of the audit conducted. Paragraph « MaximumDepth " Sets the maximum depth of the pages in which the Xenu program will be performed. Here, too, you can leave a default value. If the field « Treatredirectionsaserror " Talk, redirects will be considered an application as an error.

On the right there is a block of checkboxes « REPORT ». It serves to configure the output of the report after the site scanning. Choose those parameters whose indicators you want to see in the end:

« Broken.links,orderedbylinks » - broken links grouped by URLs;

« Broken Links, Ordered By Page " - Similarly, broken links, but grouped by pages;

« Broken.locallinks » - internal broken links;

« Redirected.URLS » - addresses with the 301th and 302nd redirects;

« FTP.and.gopher.URLS » - URL with protocols other than HTTP;

« Valid.tEXT.URLS » - not broken (working) references;

« Site.Map ","Statistics » - Add a site map report, statistics;

« Orphan.files » - Single files, i.e. those on which there are no links.

If we want to apply these settings for all projects, you need to mark the check box « Applyto.all.jobs. "

Making Xenu settings, click "OK". The program begins to audit the site. It passes throughout the blog and determines the presence of links. If there is a need to pause the application, there are "Pause" buttons on the panel above the panel, "stop" (interrupt), "Continue". How much the analysis of the resource will last, depends on the number of pages, simultaneous streams, connecting speed and server workload. Upon completion of work, a dialog box appears that reports this and proposing to bring the report.

If we agree, Xenu generates an HTM document with the results of a real-time analysis.

Now we will analyze how to use the data obtained using Xenu Link Sleuth in the program itself. The SEO-analysis results of the site are located in several columns. « Address » Contains a list of all the URL found. In order to get more detailed information about a particular link, click on it right-click, in the context menu that appears, select "URL PROPERTIES". A window opens in which the following link characteristics are displayed:

• "Page URL" - the page on which reference is maintained;
• "Title or Link Text" - the title attribute or anchor link in the material of our blog;
• "1 Page Linking to this One" - the URL of the page where the link is located.

Next comes column « Status.» where the statuses of all links are indicated. If we want to filter broken links, it is enough to simply click on the name of the column. The non-working links highlighted in red will be grouped by the Xenu program at the top of the page.

• not found - link not found;
• no Such Host - this URL does not exist;
• no Connection - the server does not respond. In this case, you should reduce the number of simultaneous streams in the settings and repeat the scan.

Bookmark « Type " Allows you to sort files by extension type: Pictures, HTML / CSS files, etc.

In the tab « Size » Collected information about the weight of files in the KB. These data can be used to find and squeeze the most difficult images on the blog.

In a collumn « Title " You can sort the results in the Title metau tag of the same name. This makes it possible to find duplicate pages on the visual coincidence of the headlines.

Also, the application provides information on such parameters:

« Level » - the depth of the pages' nesting on which the URL is located;

« Out.Links » - the number of outgoing references on a specific page;

« IN.Links » - Incoming links leading to a specific page.

Using the Xenu Link Sleuth instructions and analyzing the SEO analysis results, you can easily eliminate all broken links and most of the bang on your blog.

Netpeak Spider: Defining the main seo-parameters of the site.

This program is an analogue of Xenu. It allows you to scan the site for errors associated with internal optimization. At the same time, the application has a Russian-speaking interface, which greatly simplifies work with it. And if you are already familiar with the Xenu program, the Netpeak Spider settings for you will not be difficult.

Here is a list of those parameters that can be obtained as a result of the site audit using this program:

• « URL »- address of the scanned page;
• "Depth" - Page nesting level;
• « LinkCanonical » - the presence of the REL \u003d "canonical" attribute and the address to which it indicates;
• "Answer" - server response (3xx-5xx);
• « Title ","Description ","Keywords » - the content of the respective meta tags;
• « robots.tXT " - allowed or prohibited (allowed / disallowed) page indexing;
• « Metarobots " - availability of meta tag "Robots";
• "Redirects" - the number of redirects from the page;
• "Headers (H1) " - the number of first-level headers;
• "Links from this page", "Links to this page" - the number of all links from this page and lead to this page, respectively;
• "Internal Links";
• "External links";
• « PR ",« PR (without end-to-end links)» - Internal indicator Page Rank Pages, Internal Page Rank Page Accounting Cell Controls;
• "Duplica PoTitle "," Drokes on Description "," DublbyKeywords » - Groups of pages with a corresponding repeating meta tag.

You can download the program by reference - netpeak.ua/Software/netpeak-spider.. The application is free, but you need to go through a simple registration. After that, the activation key will be sent to the email, which must be used when installing the program. The installation process itself is standard: run file.exe and follow the instructions of the Netpeak Spider.

After starting the application, open the window with the necessary settings: "Settings" - "Scan". Select the necessary parameters, save.

In the main program window, we prescribe the address of the resource being checked, indicate the number of parallel streams and the depth of the pages. Run scanning (pictogram to the right of the field to insert the URL).

The results obtained can be filtered by the same principle as in the XENU program. To access detailed information about the indicator you are interested in, it is enough to call the context menu by clicking the right mouse button. So, for example, if we want to see a list of all outgoing links on a specific page, click on the quantity and select the "View all links from this page" in the window that appears.

To search for the doubles to the right of the main window is placed smaller. Here it is proposed to "find a duplicate" button and the ability to select filtering criteria - "by title", "on description" or "on keywords". The audit results can be saved as an Excel file. To do this, click on the main menu in the top area "Export to Excel" window.

In conclusion, you should mention that Xenu Link Sleuth and Netpeak Spider have proven themselves as excellent tools for checking broken links and have many positive feedback among webmasters. I note that although the Netpeak Spider functionality is somewhat wider, unlike Xenu, it does not have the ability to re-open the file being repeated.

Xenu's Link Sleuth is an old reliable crawler that quickly scatters the site and find literally everything: broken links, redirects, mistakes and more.

When checking the site it is important to find and correct all errors, broken links and incorrect redirects, as their absence is positive factors in the ranking of the site.

I want to advise excellent software: - Old reliable crawler, who quickly scanning the site and find literally everyone: Bought links, redirects, errors, sewn URL, document size, date and more. The latest version of 2010, but this does not prevent Xenu more mistakes in less than many modern analogues.

But this is not all, the program has many additional settings, for example:

• We can adjust the speed of analysis and the number of threads
• Display more detailed statistics on each page of the site. For example, to show all redirects, you need to go to "Options" - "Preferences" and put a tick "Treat Redirections AS Errors" - then all redirects will be displayed as an error and give answers 301 or 302:

• In Xenu, we can create a static HTML and XML site map
• Convert report to Excel format and sort the obtained data

At the last point we will stop in more detail. To open a report to Excel, you need to export the "File" report - "Export to Tab Separated File":

We will receive a report in the usual .TXT file - it needs to open NotePad ++ (or analog) and change the encoding on ANSI:

After that, you can safely open the file in Excel:

Where we can already sort the data as needed: by the date of the appearance of a document, by the number of external references, by the level of nesting page, etc.

The so-called "broken links" (English "may appear on each site. broken Links ") who lead to non-existent web pages both on this site and on other sites on the Internet.

"Bitter" or "Dead Link" may appear after changing the site structure (deletion or change of page addresses or entire partitions), while some of the pages remain the link indicating the old page address. Such links can be as "internal" (leading to non-existent pages of the same site), and "external" (leading to non-existent pages of another site). And in that and in another case, the task of the webmaster is to find and correct such links in order not to mislead visitors.

If there are few pages on the site and there are no more than 2-3 links on each, you can periodically check manually. But if there are hundreds of pages on the site, each of which has a dozen links, such a check can turn into a problem. In this case, the palate links are better to do this special program. About one of these programs called Xenu "S Link Sleuth I want to tell you.

Running program

After clicking on the program icon, its main menu appears.

To check the site, press the Ctrl + N keys or click on the edge left button "Check URL". A window opens, where we specify the address of the site that must be checked on "broken links". If the site's address ends with the catalog name, do not forget to add a "/" character at the end of the string or the entire parent directory will be checked. For example:

Right: http://www.host.com/user/

Wrong: http://www.host.com/user.

Software settings:

• Local File - Select a saved web page to check.
• Check External Links. - In addition to the "internal" all "external" links will be checked.
• Consider URL "S Beginning WITH THIS AS" INTERNAL " - Add a list of links that the program will consider "internal". Enter each such link is completed by pressing the "Add" button.
• Do Not Check Any URL "S Beginning With This - Add a list of links or partitions that the program will not check. Enter each such link is completed by pressing the "Add" button.
• More Options. - Open a window with optional settings (see below).

After clamping the "OK" button, the search for broken links will begin, which may occupy, depending on the number of page pages, the speed of their download and the number of links to each of them, from a few seconds to several hours.

The screen appears on the link found on the site, each of which is highlighted in a specific color:

• gray - The link is in the queue for checking or excluded from the verification in accordance with the specified settings.
• green - Link available
• red - Found "Dead" link or link is temporarily unavailable.

You can navigate through the list using the cursor keys (up, down, pagep, pagedown) or the scroll bar on the right.

At the bottom of the window in the status bar displays the total number of found and processed links and the time passed from the beginning of the check.

At the end of the verification, the appropriate message is displayed and the report formation request:

Advanced program settings:

• Parallel Threads. - the number of parallel to the requested pages (1 - 3 recommended, too much value can lead to a significant load on the server and slowing the page loading speed)
• Ask for Password or Certificate WHEN NEEDED - If necessary, request passwords and certificates (used for secure sites and sections)
• Treat Redirections AS Errors - count redirects errors
• Report - information that needs to be displayed in the report (you can specify the paragraphs according to the figure)
  • Broken Links, Ordered by Links - Show "Bied" links sorted to links
  • Broken Links, Ordered by Page - Show "Bied" links sorted to page
  • Broken Local Links. - Show "Bied" Local Links
  • Redirected URLs. - Displays with redirects
  • Site map - Build an HTML site map (with a large number of pages being checked, can lead to significant time delays in the formation of the report)
  • Statistics - display a different statistical information

The received report opens in your browser and consists of sections that were specified in the program settings:

If the output of statistical information was specified, then the corresponding block is added.

IN statistics The number of verified pages with a grouping by their types (HTML pages, images, scripts, files, etc.), the total size of pages are indicated. Also indicate the number of references found with grouping by their types:

• oK. - the page is found;
• skip external - the page is skipped as "external";
• forbidden Request - access to the page is prohibited;
• nOT FOUND. - Page not found.