wget: Time-Stamping Usage
1
1 5.1 Time-Stamping Usage
1 =======================
1
1 The usage of time-stamping is simple. Say you would like to download a
1 file so that it keeps its date of modification.
1
1 wget -S http://www.gnu.ai.mit.edu/
1
1 A simple ‘ls -l’ shows that the time stamp on the local file equals
1 the state of the ‘Last-Modified’ header, as returned by the server. As
1 you can see, the time-stamping info is preserved locally, even without
1 ‘-N’ (at least for HTTP).
1
1 Several days later, you would like Wget to check if the remote file
1 has changed, and download it if it has.
1
1 wget -N http://www.gnu.ai.mit.edu/
1
1 Wget will ask the server for the last-modified date. If the local
1 file has the same timestamp as the server, or a newer one, the remote
1 file will not be re-fetched. However, if the remote file is more
1 recent, Wget will proceed to fetch it.
1
1 The same goes for FTP. For example:
1
1 wget "ftp://ftp.ifi.uio.no/pub/emacs/gnus/*"
1
1 (The quotes around that URL are to prevent the shell from trying to
1 interpret the ‘*’.)
1
1 After download, a local directory listing will show that the
1 timestamps match those on the remote server. Reissuing the command with
1 ‘-N’ will make Wget re-fetch _only_ the files that have been modified
1 since the last download.
1
1 If you wished to mirror the GNU archive every week, you would use a
1 command like the following, weekly:
1
1 wget --timestamping -r ftp://ftp.gnu.org/pub/gnu/
1
1 Note that time-stamping will only work for files for which the server
1 gives a timestamp. For HTTP, this depends on getting a ‘Last-Modified’
1 header. For FTP, this depends on getting a directory listing with dates
1 in a format that Wget can parse (⇒FTP Time-Stamping Internals).
1