Downloading Web Files Without A Browser

This evening I’m relaxing and having a play with downloading web files without the need for a web browser, writing my notes as I go.

POWERSHELL
Powershell allows for an webrequest to save (or outfile) the requested file to a local location. My early attempt at a PowerShell script to do this is:

#Two variables are needed

#The first variable is where the file is online

$onlinelocation = “webaddress/images/example.png”

#The seconds variable is where the file should be saved (downloaded) to

$downloadto = “C:\temp\example.png”

#Then invoke a webrequest and utilise outfile to save the files

Invoke-WebRequest $onlinelocation -OutFile $downloadto

The draw back is that the save to folder (in this case \temp\) has to exist otherwise PowerShell errors. I’ll look at this at another time so that PowerShell checks for the save location and creates it if it does not exist.

BASH

Bash has a command called wget which is great for downloading web files:

wget webaddress/example.html

wget also has arguments that can be used before the web address:

wget -p -k webaddress/example.html downloads the page, any images it contains and the CSS referenced by the page.

wget -r -l webaddress downloads the entire website.

 

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.