18 Commits

Author	SHA1	Message	Date
CrazyMax	09c9f51bcf	remove most of absolute urls ... Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>	2022-08-03 11:50:33 +02:00
CrazyMax	ef40cfffc3	allow search engines from crawling only on production ... Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>	2022-07-19 14:11:46 +02:00
Sebastiaan van Stijn	36d0938bb2	Cleanup robots.txt ... Signed-off-by: Sebastiaan van Stijn <github@gone.nl>	2020-07-02 23:36:57 +02:00
Olly P	e63096e967	Hide the Edge Archive Image (#10246)	2020-02-06 11:49:14 +00:00
Olly P	9d919ee4e3	Added Missing Archives to robot.txt (#10029)	2019-12-13 14:20:01 +00:00
Ally Smith	736d2c76be	3.0 release configs (#9086) ... edit configs for 3.0 release	2019-07-22 17:42:56 +01:00
Maria Bermudez	18b804bbd6	Updating config, docsarchive, robots.txt files	2018-09-11 09:56:13 -07:00
Jim Galasyn	f21077adc3	Update for v17.06 archive (#4735)	2017-09-27 18:06:20 -07:00
Joao Fernandes	dd15ca212a	Update robots and configs for DDC austin	2017-08-15 23:31:15 -07:00
Misty Stanley-Jones	103a3f8003	17.03 archive setup (#3753)	2017-06-29 16:55:18 -07:00
John Mulhausen	68a969626e	Publish v1.13 archive	2017-03-02 13:48:57 -08:00
Joao Fernandes	c70a3d4636	Updates robots, sitemap for DDC	2017-02-08 21:02:34 -08:00
John Mulhausen	64d8cea742	Update robots.txt	2017-02-08 14:23:39 -08:00
Joao Fernandes	28ef624eee	Fix wrong entry for DTR in robots.txt	2016-12-21 15:57:05 -08:00
Joao Fernandes	b154bb1152	Remove old datacenter docs from sitemap, robots.txt	2016-11-10 14:11:49 -08:00
John Mulhausen	482007ee35	Creating robots.txt ahead of time ... Soon, versioned docs are coming back to subdirectories; making sure they aren't indexed.	2016-10-27 12:34:58 -07:00
John Mulhausen	a30aa7d320	Delete robots.txt	2016-10-10 11:41:32 -07:00
Misty Stanley-Jones	d53c6798c9	Convert TOML to YAML, tweaks to work with Jekyll	2016-09-29 17:16:03 -07:00