Isaac Lewis

Secure Scuttlebutt is a social network where you and your friends share your posts between your own computers. When I write a post, it's stored on my own computer in a folder. When I connect to the network, I download all the new posts from all my friends, and my friends download my new post to their computer.

Secure Scuttlebutt (SSB for short) was built by a bunch of folks in New Zealand, but started when a guy called Dominic Tarr wanted a good way to stay connected to people from his sailboat. Since the internet isn't always available, he built a system that updated when he was in port, and kept a local copy when he was out to sea. SSB was born out of those experiments, and has been used by all sorts of off-grid, offline folks to stay in touch with each other.

With SSB, you don't have to worry about a company like Facebook taking control of what you see or who sees your posts. There's no middle-man between you and your friends – your friends are the network, and you trust them instead of a company.

To get on SSB, you download an app to your computer or phone. This app will create an .ssb folder where all your posts and all your friends' posts will live. It will also create your private and public keys, which will keep your identity safe, your private messages private, and ensure that nobody can impersonate you on the network.

The app I use is called Patchwork, and it's one of the original SSB apps out there. It runs on my laptop. There's also an Android app called Manyverse. Both are pretty easy to install.

Once you've got an app installed, you'll need to find some friends! SSB works without any connection to the outside internet. When two SSB apps are on the same network, they will find each other and share content – just like gossipping. The content that's shared will stay on your device, and you can read it whenever you want. But what if you don't know anybody with an SSB app, how do you make new friends? You need to find an internet Pub to meet some new folks.

SSB Pubs are just like regular pubs – they're internet places to meet new SSB friends, and stay connected with old ones. You need an invitation to join a pub, but once you're in, you'll be able to make friends and download posts from them over the internet. You can find a list of open-invitation pubs on the SSB wiki.

SSB isn't actually a social network though. It's really a gossip protocol for sharing information that's stored and shared by your peers in turn. People have built social networks, correspondence chess games, music publishing apps, code repositories, and tons of other applications on top of SSB. There's a whole decent(ralized) world out there to explore.

Oh and when you get online, find me at the address below. Just paste it into the search box:

@+aQL1YGSyeEIhPRMcw96/ztOy41RoKSGD7la7BUZVmA=.ed25519

<3, Ike

GoAccess is a slick log viewer which allows you to drill down into web server log files easily and quickly. However, different IIS versions implement different columns in their log files, which can make parsing difficult. A little pre-processing and column mapping later, we've got our IIS server logs running correctly.

GoAccess can take raw log text as piped input. I like this approach since it allows you to manipulate your log files slightly before visualizing them, but you don't have to change your log files themselves. I used awk to remove all the comment lines from the logfiles as they were consumed.

awk -F='\n' '!/^($|[:space:]*#)/{print $1}' *.log

Here, we're grabbing every line in every log file in the current directory and unless it begins with whitespace or the "#" character, we print it. We'll save this command for later – it will be used to prepare the log data for GoAccess.

Next, we'll want to configure the column properties for GoAccess. Some IIS configurations have different column mappings or orders, so this will probably be custom for you. Here's how I created my mapping.

First, I started with this excellent script from soheilpro on GitHub. Essentially, this allows you to explicitly map each column (or "field") in your IIS logs to the appropriate code consumed by GoAccess. The final configuration will look something like this:

%d %t %^ %m %U %q %^ %^ %h %u %R %s %^ %^ %L

GoAccess will use this configuration to understand how many columns there are, and what the mapping for each column should be. The mappings are documented in the GoAccess manpage.

To use this script, I first copied the "Fields" line from one of my IIS log files. It looked like this for me:

#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status time-taken

This line is just a list of fields in order as they appear in the logfile. Many of these have a mapping code that GoAccess will use. For example, the field cs-method will map to the GoAccess code for "method": %m.

I modified the script from soheilpro to make sure that all my logs' Field values had a mapping. The modified script looked like this for me:

#!/usr/bin/env sh
while read line; do
  if [[ $line == \#Fields:* ]]; then
    line=${line/\#Fields: /}
    line=${line/date/%d}
    line=${line/time/%t}
    line=${line/s-ip/%^}
    line=${line/cs-method/%m}
    line=${line/cs-uri-stem/%U}
    line=${line/cs-uri-query/%q}
    line=${line/s-port/%^}
    line=${line/cs-username/%^}
    line=${line/c-ip/%h}
    line=${line/cs(User-Agent)/%u}
    line=${line/cs(Referer)/%R}
    line=${line/sc-status/%s}
    line=${line/sc-substatus/%^}
    line=${line/sc-win32-status/%^}
    line=${line/time-taken/%L}
    echo $line
    exit;
  fi
done

After the script was modified, I ran it against one of my log files to get the resulting GoAccess configuration:

cat u_ex190201.log | ../go-access-configuration.sh

This gave me the configuration we saw earlier:

%d %t %^ %m %U %q %^ %^ %h %u %R %s %^ %^ %L

Now we have all the pieces, and we can pipe them together. We'll first use the awk command to remove all the comments from our log files, then pipe the raw log text into the GoAccess command which will use the configuration string we just generated. Viola:

awk -F='\n' '!/^($|[:space:]*#)/{print $1}' *.log | goaccess - -o report.html --log-format "%d %t %^ %m %U %q %^ %^ %h %u %R %s %^ %^ %L" --date-format "%Y-%m-%d" --time-format "%H:%M:%S"

You'll then have a nice report ready in a report.html file in your current directory.

Kudos to the GoAccess team for a great tool!

~ <3 Ike

Let's say you want to build a website, but you want to host it yourself. You could build your own web server, host your website on a Virtual Private Server, on a RaspberryPI, or on your laptop.

But what if you could host your website without any dedicated server at all? What if you could publish your website from your computer, and have your website visitors help you host it? What if your friends could help you host your website from their computers? It sounds crazy, but it's completely possible using a new (ish) protocol called Dat.

To understand Dat, we first need to know about how a regular web server works. Most websites live on a web server. A web server holds all the files for your website, and when someone goes to your website from their computer's browser, the web pages are served using a protocol called HTTP. You don't need to know much about HTTP right now, but the important thing to know is that HTTP requires two things, a server and a client. The server is the web server you install on a VPS, a RaspberryPI, or your laptop. The client is your website visitor, using a computer and a web browser.

Dat is an alternative to HTTP. When you go to a Dat site, your browser looks for anybody on the internet who also has the files for the Dat site. If someone has the files, your browser asks them to share the files so you can load the website.

This means that anyone can host a Dat site. You can publish one right now on your laptop, and anybody else who visits your Dat site will download the files directly from your laptop.

Because this is a new way of browsing websites, not all browsers support Dat sites. Firefox, Chrome, and Safari won't know how to look around the internet and find where the files are, so they won't be able to load a Dat site. This is where Beaker Browser comes in – the new experimental browser for reading Dat sites.

When you install Beaker, you get the ability to read Dat websites. For example, here's the Dat site for the Dat project itself:

dat://60c525b5589a5099aa3610a8ee550dcd454c3e118f7ac93b7d41b6b850272330

You'll notice that this doesn't look like a web address! This is just a long string of numbers and letters – super hard to remember, and kind of ugly. This is called the public key, and it's used to find out who has the files for this site and also to make sure no one has changed the files since they were created. A bunch of awesome cryptographic math is used to create this public key.

It's not the only way to access a Dat site though. Using some DNS magic, you can make a human-readable link that uses the same public key under the hood. Here's the human-readable (and easier to remember) address for the Dat project:

dat://datproject.org/

If you click on that link in a normal browser, you'll get an error, or maybe your browser will ask you if you want to open the link with another program. But if you open the link in Beaker, you'll ask the internet who has the files for the Dat Project website, and other computers all over the world will send you files until you can load up the webpage:

That's the magic of Dat.