Another web scraping example.
A nice script to do simple web scraping and how to read json with the command-line.
A nice script to do simple web scraping and how to read json with the command-line.
How to scrape content from a web site and properly format the resulting output.
How to do simple web scraping on Linux and Windows easily. This is very useful.
A couple of very useful web scraping examples to try with Python.
A few web scraping examples for Powershell. How to get news headlines from a website.
A very useful BIOS simulator page I found, this is for Lenovo machines.
There are a lot of very useful userscripts on the Internet, these are just a few of the most useful ones. No YouTube Volume Normalization. https://gist.github.com/abec2304/2782f4fc47f9d010dfaab00f25e69c8a. This userscript will perform volume normalization on Youtube videos, this will help out when you are using Youtube for Vevo. Web Comic reader. https://github.com/ameboide/webcomic_reader/blob/master/webcomic_reader.user.js. Pretty good for reading webcomics … Read more
Parsing Apache logs is a lot of fun with awk on Linux. This can be very interesting, to see all of the bots and other visitors you get, there are so many crawlers accessing your website and indexing all of your pages. But this can be used to find unwanted leaching of bandwidth as well, … Read more
The many websites I used to enjoy like forums are vanishing and are not all being archived. This is a pity, especially if the forums had various solutions on them that another forum does not have, then the information is lost forever. But this is the evolution of the Internet. The sharing of information is … Read more
This is a nice web scraper that will read a 4chan board and return a listing of all threads on that board page. This could be very useful code to expand into a useful script. #!/usr/bin/env bash set -e links=( $( wget "$@" -qo /dev/null -SO – | grep -oE ‘</span><a href=\"thread\/[0-9]+\"’ | … Read more
These easy to use Google Searches make finding insecure web servers very easy. This one finds servers with publicly accessible pages that show web site status. They even have a complete listing of all ports and whether they are open or not! intitle:"web server status" SSH Telnetintitle:"web server status" SSH Telnet Why would a web … Read more
SSL packets sent to and fro from your machine can contain a lot of useful information. I am going to show how to see what websites are visited on a network. Firstly, load Wireshark and then listen to traffic on your machine. Then visit a few different websites with Firefox or Chrome. Use this filter … Read more
Amazon Web Services is the perfect solution for hosting a server outside your organization. This allows you to create a cloud hosted database server that takes the strain off your local server infrastructure using Microsoft SQL server 2012 or a Linux based solution in a cloud hosted server instance. There is a free trial that … Read more
Scanning for vulnerable ports is easy with the Kali Linux distribution. Here is a simple port scan of my Windows Server 2012 R2 laptop host OS using Kali Linux. homer@kali:~$ sudo nmap 192.168.1.6 Starting Nmap 6.46 ( http://nmap.org ) at 2014-05-28 20:44 EST Nmap scan report for 192.168.1.6 Host is up (1.0s latency). Not … Read more
How to get the geographical location of an IP address. This simple command will get this for you. [jason@darknet:~] curl ipinfo.io/203.113.124.148 ; echo "" { "ip": "203.113.124.148", "hostname": "No Hostname", "city": "", "region": "", "country": "TH", "loc": "13.7500,100.4667", "org": "AS9737 TOT Public Company Limited" }[jason@darknet:~] curl ipinfo.io/203.113.124.148 ; echo "" { "ip": "203.113.124.148", "hostname": "No … Read more
The new Apple SSL vulnerability is a concerning bug in the Apple Macintosh iOS and desktop operating systems. This allows SSL spoofing as it will allow a faked SSL vulnerability to pass as a real one. Apparently it uses ports 1266 & 1267, if you block these with your firewall, this should alleviate this threat … Read more
http://www.therestartpage.com/#. This is a website that has shutdown dialogs for a host of operating systems. Click restart on one of them and you can watch the restart animation and/or POST for that OS. This is one very cool website. The ability to see what the boot animation for Windows `95 and 2000 looked like really … Read more
The onslaught of technological innovations over the last decade posed an enormous challenge to web developers in adapting to the various devices, platforms and browsers. Every day we see a new smart phone, tablet and notebooks being launched and this mad invasion of Technology is only meant to grow bigger by the day. Why Responsive … Read more
http://info.cern.ch/hypertext/WWW/TheProject.html. This is the first Hypertext page ever created; the page that was the first to use hyperlinks and gave us a connected and more informed future. This is a page using some very old HTML 1.0 code with<DD> tags that I have not seen for a very long time. They seem to have gone … Read more