valentineus/popov.link
0
Fork 0
mirror of https://github.com/valentineus/popov.link.git synced 2025-11-04 06:49:45 +03:00
Code Issues Releases Activity

Flexible exception list

Browse Source 
Signed-off-by: Valentin Popov <info@valentineus.link>
This commit is contained in:
Valentin Popov
2018-05-15 15:53:48 +04:00
parent f0e1ca0b33
commit 7aff2fd186
2 changed files with 6 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
_data/robots_disallow.yml Normal file
View File

@@ -0,0 +1 @@
- path: "/404.html"

5
_pages/robots.txt
View File

@@ -5,6 +5,9 @@ layout: null
User-agent: * User-agent: *
Allow: {{ site.baseurl | relative_url | uri_escape }} Allow: {{ site.baseurl | relative_url | uri_escape }}
Disallow: {{ "/404.html" | relative_url | uri_escape }}
{% for link in site.data.robots_disallow %}
Disallow: {{ link.path | relative_url | uri_escape }}
{% endfor %}
Sitemap: {{ "sitemap.xml" | absolute_url | uri_escape }} Sitemap: {{ "sitemap.xml" | absolute_url | uri_escape }}