Downloading all files from URL Folder to local folder
I want to download all contents from a url using Windows 10, like
This folder may contain 100 files, and I want to put them all into
C:\backupsI am going to schedule this script to run once every hour, and I don't want it to download the files that already exists.
I've tried a few scripts, like:
bitsadmin.exe /transfer "test" C:\backupsWithout any success, note: I have a very small experience with batch scripts
22 Answers
You will find a couple of useful PowerShell scripts in the postHow to download all files from URL?
Here is one of the two:
$outputdir = 'D:\Downloads'
$url = '
# enable TLS 1.2 and TLS 1.1 protocols
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12, [Net.SecurityProtocolType]::Tls11
$WebResponse = Invoke-WebRequest -Uri $url
# get the list of links, skip the first one ("../") and download the files
$WebResponse.Links | Select-Object -ExpandProperty href -Skip 1 | ForEach-Object { Write-Host "Downloading file '$_'" $filePath = Join-Path -Path $outputdir -ChildPath $_ $fileUrl = '{0}/{1}' -f $url.TrimEnd('/'), $_ Invoke-WebRequest -Uri $fileUrl -OutFile $filePath
}There are quite a few ways to handle this and as the other answer here provides, you can do this with Powershell if strictly doing it with built-in Windows 10 functions.
Alternately, I do exactly what you are talking about using the powerful, free and (reasonably) intuitive program,
It is designed for crawling and copying websites locally, but you can configure filters or custom URL lists so that it only grabs certain URL paths or certain file types. In my case I have it track a website and download all the weather map .jpg files that update 4 times a day.
Once it's configured and successfully run, you can then use httrack's command line capabilities to continue to update/mirror the site/url/files in a .bat file and then run it as a scheduled task in Windows 10. The .bat file that you set to activate via the Windows Task Scheduler would look something like this.
set path="C:\Program Files (x86)\WinHTTrack";%path%
cd D:\Backups\WebsiteProjectFolder\
httrack --update
