Downloading Web Files Without A Browser

This evening I’m relaxing and having a play with downloading web files without the need for a web browser, writing my notes as I go.

POWERSHELL
Powershell allows for an webrequest to save (or outfile) the requested file to a local location. My early attempt at a PowerShell script to do this is:

#Two variables are needed

#The first variable is where the file is online

$onlinelocation = “webaddress/images/example.png”

#The seconds variable is where the file should be saved (downloaded) to

$downloadto = “C:\temp\example.png”

#Then invoke a webrequest and utilise outfile to save the files

Invoke-WebRequest $onlinelocation -OutFile $downloadto

The draw back is that the save to folder (in this case \temp\) has to exist otherwise PowerShell errors. I’ll look at this at another time so that PowerShell checks for the save location and creates it if it does not exist.

BASH

Bash has a command called wget which is great for downloading web files:

wget webaddress/example.html

wget also has arguments that can be used before the web address:

wget -p -k webaddress/example.html downloads the page, any images it contains and the CSS referenced by the page.

wget -r -l webaddress downloads the entire website.