Privacy (confidentiality) of data is of great importance, especially in projects such as distance education based on Moodle, because administrator has to upload a lot of inside information: orders, lists, guidelines, certifications and so forth. Agree, this information is intended only for only some people, ie, for internal use and leakage of information is not only desirable, but also, in some cases, very simply unacceptable, for example, if its a paid content / courses. In any case, the internal data is better to hide, even if at this point you do not have a large amount of content / files online LMS Moodle.
How your private files can be found
Search engine spiders index the huge amount of information that can be “wrong to lie.” This means that search engines do not care what content to index, is only some reference to the private information getting caught his eye. There are several examples where the information about the elite foreign real estate Russian government officials indexed at Google, probably the homeland there seems better. It does not have to be a “hacker” to get this information, we need only examine the results of a query in Google: site:needlesite.ru.
What to do?
All search engines comply with the rules set in the file robots.txt, lying at the root of the site, for example: https://lms-service.org/robots.txt. If this file is not present, so prohibitions for indexing site either, but if the file exists, then the robot to index the content according to the rules described therein.
For the site based on Moodle is quite difficult to write a universal file robots.txt, because it all depends on the installed modules, as well as the willingness of the site administrator to index a particular content. For example, you have a forum module installed and working, but if you want to index the user messages in it? On the one hand, as a rule, nothing secret at the forum there, and once indexed content, as a result, increase search engine traffic, on the other hand, some administrators do not want to “wash dirty linen in public.” Therefore, briefly explain the principle of the file robots.txt.
Creation
By file extension .txt is clear that this is a plain text file, so you can create it with any text editor, it is preferable to use a program like Notepad++ or Notepad2. Also optional but desirable to file encoding was UTF-8.
Filling
Basically, there are no differences for the construction of the file under the different search engines all have the same rules, but Yandex has its Host directive on which it determines the main mirror site, so it’s best to separate rules for Yandex and for all the other search engines, because due of the Directive in the search engines can be a problem.
To start close by indexing the site and discover just some sections and Home:
User-agent: Yandex Disallow: / Allow: /$ Allow: index.php$ Allow: /course/info.php?id= Allow: /course/view.php?id=
There is such a principle: first, prohibiting all and then allow something individually, for example, in the above example, we allowed only index information about the courses and index page.
Long gathering information, we came to a more or less universal option settings robots.txt for Moodle, maybe we missed something or you do not like file access to certain sections, so you can correct it at any time. So what happened:
User-agent: Yandex Disallow: / Allow: /$ Allow: /index.php$ Allow: /calendar/view.php Allow: /course/info.php?id= Allow: /course/view.php?id= Allow: /mod/page/view.php?id= Allow: /mod/forum/view.php?id= Clean-param: search /mod/forum/view.php Clean-param: showall /mod/forum/view.php Clean-param: mode /mod/forum/view.php Allow: /mod/forum/discuss.php?d= Allow: /mod/wiki/view.php?id= Allow: /pix Allow: /user/view.php?id= Host: your-moodle-site.com User-agent: * Disallow: / Allow: /$ Allow: /index.php$ Allow: /calendar/view.php Allow: /course/info.php?id= Allow: /course/view.php?id= Allow: /mod/page/view.php?id= Allow: /mod/forum/view.php?id= Clean-param: search /mod/forum/view.php Clean-param: showall /mod/forum/view.php Clean-param: mode /mod/forum/view.php Allow: /mod/forum/discuss.php?d= Allow: /mod/wiki/view.php?id= Allow: /pix Allow: /user/view.php?id=
It is important to observe the line breaks!
