Read the news headlines from the Daily Telegraph using Powershell.

Web scraping with Powershell is very easy. This example allows a simple method of scraping all headlines from the website using only Powershell.

PS C:\Users\Intel i5> ([regex]::Matches((Invoke-RestMethod "https://www.dailytelegraph.com.au/news/nsw"), '(?<=data-tgev-container="feed-1">).*?(?=</a>)') | ForEach-Object { $_.Value })
Man shot dead at Bondi Junction in suspected gang hit
Man shot dead at Bondi Junction in suspected gang hit
Burnt out car found at James St in Zetland may be linked to Bondi Junction shooting
‘Class War’: Shocking data reveals NSW’s most violent schools
Complete guide: Top events, shows in NSW this year
Six Sydney suburbs to cop the most noise from new airport
Daily Telegraph &#8211; News Feed
‘Why kids are arming themselves at school’: Expert
Fate decided on Meriton’s massive Zetland development
Big names axed from Sydney’s power list
‘Vital’: CBD night-life boost with precinct proposals
I’m Offended: comedian Vince Sorrenti hits out at woke culture
Shock number of ChatGPT school cheats revealed
Watch shock footage of violent new trend in NSW schools
Guy Sebastian’s ex-manager has bail win
The kind of thinking that has real potential
Gifted gowns: Relief for brides with dresses lost in liquidation
Caringbah teacher’s assault charges dismissed
First look inside Channel 7’s new Sydney HQ
Pistol, ammo, drugs allegedly seized at west Sydney property
Daily Telegraph 2023 Sydney Power 100 list
Six men have been charged over an interstate drug trafficking ring
‘I won’t pay’: Fitness trainers fear closure if fee hike approved
Revealed: Plans to rebuild servo after fuel tanker explosion
Fresh charge against Berejiklian’s ex-beau Daryl Maguire

Given this example.

<h4 class="storyblock_title"><a class="storyblock_title_link" href="https://www.dailytelegraph.com.au/newslocal/st-george-shire/ben-weldon-caringbah-teachers-common-assault-charges-dismissed/news-story/dd75d6458c1171d5f5b016d69f832119" data-tgev="event10" data-tgev-metric="npv" data-tgev-order="19" data-tgev-label="story" data-word-match="" data-tgev-container="feed-1">Caringbah teacher’s assault charges dismissed</a></h4>

It is very easy to get the text from between the data-tgev-container=”feed-1″> and </a> tags. This is a great example of web scraping that would be easy to adapt to any website.

Here is another example.

PS C:\Users\Intel i5> (Invoke-RestMethod "https://api.openweathermap.org/data/2.5/weather?q=Detroit&appid=166580d2219dfb34f77ac03b88820246&units=metric")
 
 
coord      : @{lon=-83.0458; lat=42.3314}
weather    : {@{id=804; main=Clouds; description=overcast clouds; icon=04n}}
base       : stations
main       : @{temp=19.4; feels_like=19.9; temp_min=18.5; temp_max=20.53; pressure=1000; humidity=96}
visibility : 10000
wind       : @{speed=7.2; deg=280; gust=9.77}
clouds     : @{all=100}
dt         : 1687828451
sys        : @{type=2; id=2006979; country=US; sunrise=1687773412; sunset=1687828376}
timezone   : -14400
id         : 4990729
name       : Detroit
cod        : 200

Getting the current weather in Detroit.

And yet another web scraping example. Getting all news headlines from the Herald Sun website.

PS C:\Users\Intel i5> ([regex]::Matches((Invoke-RestMethod "https://www.heraldsun.com.au/news/national"), '(?<=<p class="storyblock_standfirst g_font-body-s">).*?(?=</p>)') | ForEach-Object { $_.Value })
Australia’s largest health insurer will undergo an internal risk and governance review after it suffered the largest data breach in the country’s history.
Unscrupulous NDIS providers have been accused of ripping off vulnerable workers, with the Commonwealth minister issuing a stern warning to businesses.
The billionaire businesswoman has made a bold foray into the British beef market as the first big mover under the recently signed Australia-UK free trade deal.
Ukraine has asked Australia for one thing for months, but it wasn’t included in the recently announced $110 million assistance package.
A new report has revealed the grim reality of the cost of living crisis, with thousands struggling to pay their bills and cutting down on heating and power use.
Guy Sebastian’s former manager Titus Day has been granted bail by the Supreme Court while he awaits the outcome of his appeal against his conviction and jail sentence.
Warnings about buying tickets to Taylor Swifts’ Australian gigs have been ramped up as the first window to get seats has closed. See why and tips to get your hands on some.
PwC Australia acting CEO Kristin Stubbins is under mounting pressure to swiftly deliver the findings of its internal inquiry into the tax scandal that has engulfed the consultancy firm.
Seven’s new recruit Dr Chris Brown has another TV gig in the wings on top of his two new projects for the network.
The airline will hand back customers some of the money related to Covid-19 cancellations as the December expiry date looms.
A drug that is 50 to 100 times more potent than heroin and morphine – and turning users into zombies – could be heading to Australia’s streets, authorities warn.
One of Australia’s big four banks is experiencing a major outage to its online banking services, leaving Aussies unable to access their money.
After a chaotic weekend in Russia, Australia has pledged a new package to support Ukraine, as the conflict reaches a grim milestone
The family of former federal Labor leader Simon Crean has described him as “humble, charismatic and loved by so many”, after his sudden death in Germany at the age of 74.
The former CEO of a series of disability homes where a girl died in a bath is still making money from the NDIS, despite questions being asked. FIND OUT HOW
A report into the dating habits of 18 to 49 year old singles has revealed three in five men have been put off dating because of the cost. See what else they had to say.
Australian shares are heading for a volatile week as worries over inflation and events in Russia threaten to take the gloss off the market’s end of financial year performance.
The masterminds behind an audacious art heist which netted several prized Sidney Nolan art works have been unmasked. FIND OUT HOW
June 30 is less than a week away, so try to tick off these key tax and other issues to stay ahead of the financial pack.

PS C:\Users\Intel i5> ([regex]::Matches((Invoke-RestMethod "https://www.heraldsun.com.au/news/national"), '(?<=<p class="storyblock_standfirst g_font-body-s">).*?(?=</p>)') | ForEach-Object { $_.Value }) Australia’s largest health insurer will undergo an internal risk and governance review after it suffered the largest data breach in the country’s history. Unscrupulous NDIS providers have been accused of ripping off vulnerable workers, with the Commonwealth minister issuing a stern warning to businesses. The billionaire businesswoman has made a bold foray into the British beef market as the first big mover under the recently signed Australia-UK free trade deal. Ukraine has asked Australia for one thing for months, but it wasn’t included in the recently announced $110 million assistance package. A new report has revealed the grim reality of the cost of living crisis, with thousands struggling to pay their bills and cutting down on heating and power use. Guy Sebastian’s former manager Titus Day has been granted bail by the Supreme Court while he awaits the outcome of his appeal against his conviction and jail sentence. Warnings about buying tickets to Taylor Swifts’ Australian gigs have been ramped up as the first window to get seats has closed. See why and tips to get your hands on some. PwC Australia acting CEO Kristin Stubbins is under mounting pressure to swiftly deliver the findings of its internal inquiry into the tax scandal that has engulfed the consultancy firm. Seven’s new recruit Dr Chris Brown has another TV gig in the wings on top of his two new projects for the network. The airline will hand back customers some of the money related to Covid-19 cancellations as the December expiry date looms. A drug that is 50 to 100 times more potent than heroin and morphine – and turning users into zombies – could be heading to Australia’s streets, authorities warn. One of Australia’s big four banks is experiencing a major outage to its online banking services, leaving Aussies unable to access their money. After a chaotic weekend in Russia, Australia has pledged a new package to support Ukraine, as the conflict reaches a grim milestone The family of former federal Labor leader Simon Crean has described him as “humble, charismatic and loved by so many”, after his sudden death in Germany at the age of 74. The former CEO of a series of disability homes where a girl died in a bath is still making money from the NDIS, despite questions being asked. FIND OUT HOW A report into the dating habits of 18 to 49 year old singles has revealed three in five men have been put off dating because of the cost. See what else they had to say. Australian shares are heading for a volatile week as worries over inflation and events in Russia threaten to take the gloss off the market’s end of financial year performance. The masterminds behind an audacious art heist which netted several prized Sidney Nolan art works have been unmasked. FIND OUT HOW June 30 is less than a week away, so try to tick off these key tax and other issues to stay ahead of the financial pack.

This shows how the web scraping function of this Powershell one-liner works just fine.

Leave a Comment Cancel reply