Starbeamrainbowlabs

Stardust
Blog


Archive


Mailing List Articles Atom Feed Comments Atom Feed Twitter Reddit Facebook

Tag Cloud

3d 3d printing account algorithms android announcement architecture archives arduino artificial intelligence artix assembly async audio automation backups bash batch blender blog bookmarklet booting bug hunting c sharp c++ challenge chrome os cluster code codepen coding conundrums coding conundrums evolved command line compilers compiling compression conference conferences containerisation css dailyprogrammer data analysis debugging defining ai demystification distributed computing dns docker documentation downtime electronics email embedded systems encryption es6 features ethics event experiment external first impressions freeside future game github github gist gitlab graphics guide hardware hardware meetup holiday holidays html html5 html5 canvas infrastructure interfaces internet interoperability io.js jabber jam javascript js bin labs latex learning library linux lora low level lua maintenance manjaro minetest network networking nibriboard node.js open source operating systems optimisation outreach own your code pepperminty wiki performance phd photos php pixelbot portable privacy problem solving programming problems project projects prolog protocol protocols pseudo 3d python reddit redis reference release releases rendering research resource review rust searching secrets security series list server software sorting source code control statistics storage svg systemquery talks technical terminal textures thoughts three thing game three.js tool tutorial twitter ubuntu university update updates upgrade version control virtual reality virtualisation visual web website windows windows 10 worldeditadditions xmpp xslt

I've got some business cards!

My new business cards!

I've been to several events of various natures now (like the Hardware Meetup), and at each one I've found that a 'business card' or two would be really handy to give to people so that they can remember the address of this website.

After fiddling with the design over about a week I (and Mythdael!) came up with the design you see above. Personally, I'm really pleased with them, so I decided to post here to show them off :-)

I'm finding that it's a rather good idea to promote and build your brand at these kind of events by showing people the cool things that you've created and learnt, and business cards seem to be just the thing that helps you do it.

A close up of the front and back of my new business cards.

How to run a successful blog: My experiences so far

About 2 and a half years (and 207 posts!) ago today, I took the advice of Rob Miles and started this blog. I've learned a lot since I started from some wonderful people, and the quality of my posts on here has improved immensely. I've been meaning to write a post on what I've learnt about running this blog, but haven't gotten around to it until now :-)

The first, and most important, thing you can do is to post often. By posting often you keep people coming back to read more. If you leave it too long between posts, people may forget about your blog and may not come back.

Of course, while having regular content is good, having a posting schedule is considered by some to be even better. While I'm not particularly good at this yet, if you do manage to achieve it you can ensure that people know when they can come back and expect another post.

It's also important that you post about something you find interesting. Doing so improves the quality of the post, and it makes for a much more interesting read. If you're passionate about what you're writing about, people will generally come back for more.

Next, blogging is something that you have to persevere with. You certainly won't acquire readers overnight. It's worth bearing in mind that most of your readers will be silent - only a very small proportion of readers will actually leave a comment, no matter how easy you make the process (I hardly ever get comments).

Also, don't set out with the goal of getting $x$ number of regular readers, or earning $y$ amount of money. They are bad motivators to have, and you will surely be disappointed. Rather, you should set out to just write about something you love, without expecting anyone to read what you've written. Then you'll be pleasantly surprised :D

That's all I can think of right now. There are surely other things that I haven't mentioned. If you have any tips you've learnt from running a blog of your own, post about them in the comments :D

My PixelBot is Connected! (Part 1)

After many attempts and solving many problems, I've finally gotten my Wemos-powered PixelBot to connect via TCP to a C# server component I've written. Since I experienced so many problems with it, I decided to post about it here to help others who want to do the same as I.

The first (and arguably most difficult) hurdle I came across was the lack of correct information. For one, the TCP client class is actually called WifiClient (which is confusing in and of itself). For another, there aren't any really good tutorials out there that show you what to do.

In addition, I wanted to build a (rather complicated as it turns out!) auto discovery system to allow my PixelBot to find the PixelHub (the server) automatically. As usual, there was even less information about this task! All I found was an outdated guide and rather simplistic example. I ended up inspecting the header file of the WiFiUDP class in order to figure it out.

In this post, I'm going to explain how I got the autodiscovery mechanism working. Before we begin, you need to understand what multicast UDP is and how it works. For those of you who don't, I've posted all about it.

There are several ways that one can go about writing an autodiscovery mechanism. While Rob Miles chose mDNS, I ended up approaching the problem from a different angle and writing my own protocol. It's best explained with a diagram:

A diagram explaining the PixelBot's autodiscovery mechanism.

  1. The PixelHub server has a beacon that sends out pings to tell the PixelBots where it is
  2. The PixelBots subscribe to the beacon ping channel when they want to find the server
  3. The PixelBots decode the incoming ping packets to find the discover of the server
  4. The PixelBots connect to the PixelHub server using the credentials that it found in the beacon pings it decoded!

In order to receive these beacon pings, we need to set up a UDP listener and wire it up to receive multicast packets. In the following example I'm multicasting on 239.62.148.30 on port 5050.

IPAddress beaconAddress = IPAddress(239, 62, 148, 30);
unsigned int beaconPort = 5050;

WiFiUDP UdpClient;
UdpClient.beginMulticast(WiFi.localIP(), beaconAddress, beaconPort);

After connecting to the multicast address, we then need to receive the pings:

// Create a buffer to hold the message 
byte datagramBuffer[datagramBufferSize];
// Prefill the datagram buffer with zeros for protection later
memset(datagramBuffer, '\0', datagramBufferSize);
while(true) {
    int datagramSize = UdpClient.parsePacket();
    Serial.print("Received datagram #");
    Serial.print(datagramSize);
    Serial.print(" bytes in size from ");
    Serial.print(UdpClient.remoteIP());
    Serial.print(":");
    Serial.print(UdpClient.remotePort());

    // Don't overflow the message buffer!
    if(datagramSize > datagramBufferSize) {
        Serial.println(", but the message is larger than the datagram buffer size.");
        continue;
    }
    // Read the message in now that we've verified that it won't blow our buffer up
    UdpClient.read(datagramBuffer, datagramSize);

    // Cheat and cast the datagram to a character array
    char* datagramStr = (char*)datagramBuffer;

}

That's rather complicated for such a simple task! Anyway, now we've got our beacon ping into a character array, we can start to pick it apart. Before we do, here's what my beacon pings look like:

server@10.0.40.66:5050
  ^        ^        ^
 Role IP Address   Port

Although it looks simple, in C++ (the language of the arduino) it's rather a pain. Here's what I came up with:

// Define the role of the remote server that we're looking for
char desiredRemoteRole[7] = { 's', 'e', 'r', 'v', 'e', 'r', '\0' };

// Find the positions of the key characters
int atPos = findChar(datagramStr, '@');
int colonPos = findChar(datagramStr, ':');

// Create some variables to store things in
char role[7];
char serverIp[16];
char serverPortText[7];
int serverPort = -1;
// Fill everything with zeroes to protect ourselves
memset(role, '\0', 7);
memset(serverIp, '\0', 16);
memset(serverPortText, '\0', 7);
// Extract the parts of the 
strncpy(role, datagramStr, atPos);
strncpy(serverIp, datagramStr + atPos + 1, colonPos - atPos - 1);
strncpy(serverPortText, datagramStr + colonPos + 1, datagramSize - colonPos - 1);

Serial.println("complete.");

// Print everything out to the serial console
Serial.print("atPos: "); Serial.println(atPos);
Serial.print("colonPos: "); Serial.println(colonPos);

Serial.print("Role: "); Serial.print(role); Serial.print(" ");
Serial.print("Remote IP: "); Serial.print(serverIp); Serial.print(" ");
Serial.print("Port number: "); Serial.print(serverPortText);
Serial.println();

// If the advertiser isn't playing the role of a server, then we're not interested
if(strcmp(role, desiredRemoteRole) != 0)
{
    Serial.print("Incompatible role "); Serial.print(role); Serial.println(".");
    continue;
}
Serial.println("Role ok!");
serverPort = atoi(serverPortText);

Phew! That's a lot of code! I've tried to annotate it the best I can. The important variables in the are serverIp the serverPort - they hold the IP address and the port number of the remote machine that we want to connect to.

I hope that's somewhat helpful. If there's anything you don't understand or need help with, please post a comment down below :-)

Next time, I'm going to show off the TCP connection bit of the system. Stay tuned :D

Pepperminty Wiki Turns 2!

Pepperminty Wiki turns 2(!) :D

2 years ago today, I decided that I'd build a wiki. At the time, I was unsatisfied with all the currently solutions out there - they were either too bulky, old and unmaintained, or just otherwise not quite right. I decided to build something different: An entire wiki in a single file. One single file that you can drop onto your web server and have it just work.

Although I've had to rework and rewrite a few things along the way, development has generally gone ok. I've found that as it's grown, I've needed to change the way I design it slightly - it's been a great learning experience!

In May this year, I managed to get Pepperminty Wiki into Awesome Self Hosted, a list of cool pieces of software that you can host on your own server - a list that has ~12,000 stars on GitHub.

Fast forward to the present, and I find myself with an awesome wiki - that's still all contained in a single file. It's powered by Parsedown. It supports multiple users, page protection, sub pages, full text search (!), customisable themes, tags, redirects, and more! It's also got a whole host of configurable settings system - allowing you to customise how your wiki works to your liking. Best of all, it's got a flexible module based system - so anyone can come along and write a new module to extend it's functionality.

If this sounds like the kind of thing you'd like to use yourself, just head over to the getting your own copy section on it's page (for the lazy, the online downloader is here).

Going forwards, I'd like to a commenting system to let people comment on pages on a wiki. I'd like to add edit previews. I'd like to add a GUI for the settings file. I've got so many ideas that it's difficult to choose which to do next :D

Thank you, everyone for the last 2 years. Here's to another 2 amazing fun filled years! I don't intend to stop development any time soon :D

An easier way to debug PHP

A nice view of some trees taken by Mythdael I think, the awesome person who designed this website :D

Recently at my internship I've been writing quite a bit of PHP. The language itself is OK (I mean it does the job), but it's beginning to feel like a relic of a bygone era - especially when it comes to debugging. Up until recently I've been stuck with using echo() and var_dump() calls all over the place in order to figure out what's going on in my code - that's the equivalent of debugging your C♯ ACW with Console.WriteLine() O.o

Thankfully, whilst looking for an alternative, I found xdebug. Xdebug is like visual studio's debugging tools for C♯ (albeit a more primitive form). They allow you to add breakpoints and step though your PHP code one line at a time - inspecting the contents of variables in both the local and global scope as you go. It improves the standard error messages generated by PHP, too - adding stack traces and colour to the output in order to make it much more readable.

Best of all, I found a plugin for my primary web development editor atom. It's got some simple (ish) instructions on how to set up xdebug too - it didn't take me long to figure out how to put it to use.

I'll assume you've got PHP and Nginx already installed and configured, but this tutorial looks good (just skip over the MySQL section) if you haven't yet got it installed. This should work for other web servers and configurations too, but make sure you know where your php.ini lives.

XDebug consists of 2 components: The PHP extension for the server, and the client that's built into your editor. Firstly, you need to install the server extension. I've recorded an asciicast (terminal recording) to demonstrate the process:

(Above: An asciinema recording demonstrating how to install xdebug. Can't see it? Try viewing it on asciinema.org.)

After that's done, you should be able to simply install the client for your editor (I use php-debug for atom personally), add a breakpoint, load a php page in your web browser, and start debugging!

If you're having trouble, make sure that your server can talk directly to your local development machine. If you're sitting behind any routers or firewalls, make sure they're configured to allow traffic though on port 9000 and configured to forward it on to your machine.

Coding Conundrums Evolved 4: Prime Wall Hangings

This post is part of a biweekly series of programming problems, targeted at those learning to program using C# at University. The problems will vary in difficulty - some will be rather easy, and some will be quite challenging! Don't worry if you can't solve a problem just yet - come back to it in a few months and it'll seem a lot easier than it did before.

Welcome to episode 4 of coding conundrums evolved. If you missed the last one, it is all about fiddly fuel and is located here. The password to the archive that contains my solution for it is iveinya, but try solving it on your own first if you haven't already :-). This episode holds a bit of a different challenge for you.

The famous Professor Arkin, discoverer of illykin theory (the technology behind the quadrilateral engine first installed in the Lilai), has recently been recognised and awarded the prestigious Byema Or award. Along with this award comes a place for a statue in the hall of fame at Timiku University on the planet Eiriodin, the most distinguished and renowned University in the galaxy!

Since it's generally considered to be rather important to have an impressive statue (as you'd naturally expect!), Professor Arkin hired only the most skilled craftsmen to create his statue. After having the magnificent statue made and put in the hall, he thinks that there is still something missing.

Professor Arkin, upon hearing about your impressive skills from Chief Engineer Nebrilla, has asked you to write a program that generates a wall hanging to go behind his statue. In his specification for the wall hanging, he wrote this:

The exact nature of the wall hanging is up to you, but I am interested in something particularly impressive looking. The more impressive it looks, the more credits I will pay you for your trouble!

One of the key secrets to my illykin theory is actually prime numbers, so your design should feature them prominently.

(Author's Note: No actual funds will be paid upon completion of this challenge! Post about your creations in the comments and have a virtual cookie instead :D)

Write a program that generates a suitable wall hanging for Professor Arkin. Make sure that you feature prime numbers in your design!

Helpful Hints

  • Plan your design on paper first so that you have something to work towards.
  • There isn't any set solution for this problem (though I will certainly be providing mine in the next episode!).
  • GDI+ is the native drawing library for C#. Make sure you include a reference to System.Drawing.dll if you decide to use it!
  • I've written a simple image generator template here: ImageGenerator.cs. If you have trouble getting started, feel free to use it as a starting point!

Here's what I managed to put together. I'm sure you can do better than I :-)

My creation.

There isn't a challenge for this conundrum this time - though if you want one you could always try and animate your wall hanging (Professor Arkin is considering digital projector-based installations too). As usual the archive for my solution is here, and the password will be released in the next post.

SBRL Archives: Colour Picker

Since I've been rather ill suffering the after effects of this year's flu vaccination and I haven't finished the next post I was writing for this week, I'm posting this instead :-)

A few weeks ago I went digging through my archives and I found a few gems just lying around, so I thought I'd post about one of the things I found! This particular project is from waaay back in 2013, when I hadn't started University and learnt C# (Thanks Rob :D), and was still learning Javascript.

My colour picker from 2013.

The project in question (as you might have guessed by the title) is a simple colour picker. You can find it here:

2013 Colour Picker

I've been careful to make only minimal changes to the code before uploading it here (updating comments, switching onload out for window.addEventListener() etc.) - it's interesting to compare my programming style then to the way that I do things now, especially since I was entirely self-taught at that point in time.

Looking back now, there are several things I'd change. For one I'd remove the dependency on functions.js - an early attempt of mine to create a utility library, and rather put each method in a gist and copy the ones I need. For another I'd certainly separate the code that writes to the DOM (and update it to use a document fragment & the DOM api rather than innerHTML) from the code that performs the core logic.

It's also interesting to note that Javascript wasn't actually the language that I started with. My tale actually starts back when I was in my last year of primary school, when I discovered Game Maker by Mark Overmars, which later went on create YoYo Games (It was called Game Maker 7 back then!). After growing out of it, I found its integrated 'programming language', GML. Only after learning that did I start to investigate the technologies of the web - HTML5, CSS3, and then Javascript.

Hopefully this post has been an interesting read - I should have the next regular post ready for later this week :-)

More automatic reposting!

I've had automatic tweeting about blog posts I put up on here for ages now. I've also had a mailing list set up for a while too. Today it occurred to me that not everyone uses twitter or wants an email, and so I've just set up a few more automatic reposting systems (thanks to IFTTT!).

  • If you know me through facebook, I've set it up to post about my blog posts on my timeline.
  • If you use reddit, then I've set up a new subreddit that IFTTT will automatically post links to my new blog posts in.

That brings the total number of channels you get notified about my blog posts up to 5(!)

I'm still tweaking and perfecting the new systems - please comment if you notice anything that's off or could be improved. Also, let me know if there's a channel that I haven't covered that you'd like me to set up.

Coding Conundrums Evolved 3: Fiddly Fuel

This post is part of a biweekly series of programming problems, targeted at those learning to program using C# at University. The problems will vary in difficulty - some will be rather easy, and some will be quite challenging! Don't worry if you can't solve a problem just yet - come back to it in a few months and it'll seem a lot easier than it did before.

Rocket engines at NASA. (Above: Some 15 RS-25 rocket engines at NASA. Source: NASA Image and Video Library)

Get ready for 3rd episode of Coding Conundrums Evolved! This time, you've been asked to write some software for a brand new ftl engine.

First though, the password to the last solution is zirros.

It's the year 2252, and the engineers at the Hazuva spacecraft dry dock orbiting the ocean world Eiriodin are busily working away on putting the finishing touches to a brand new elligon class starship. Capable of FTL travel like no other craft before it, the new ship (dubbed the Lilai by the engineers) will be able to take its crew on an unforgettable journey across the stars.

There's a problem though. The Lilai has a brand new engine that was designed from scratch with a quadrilateral fuel regulation valve. As a result, the standard linear fuel balancing agent on the engine's control computer needs to be completely rewritten! Upon hearing the news, Chief Engineer Nebrilla has hired you to do the job.

The engine in question handles fuel through a grid of reactor cells. Oddly enough, the fuel in the reactor cells must be arranged in ratios such that they form a 5x5 magic square.

         
17 24 1 8 15
23 5 7 14 16
4 6 13 20 22
10 12 19 21 3
11 18 25 2 9
         

(Above: An example magic square.)

Write a program that outputs a 5x5 magic square.

Helpful Hints

  • The Siamese method can be used to programatically generate a magic square.
  • Read the description of the algorithm carefully and make sure you understand it before writing any code.
  • Follow the algorithm through on paper first.
  • You may want to take a look at using a 2D array to store information about your magic square.
  • I've written a magic square validator so that you can test your implementation.
    • Use it like this on Windows: type MagicSquare.txt | ./MagicSquareValidator.exe [Order] [MagicNumber]
    • ...or this on Linux: cat MagicSquare.txt | mono ./MagicSquareValidator.exe [Order] [MagicNumber]

Challenge

The new engine design tested in the Lilai was so successful that a second ship, the Iveinya, is going to be retrofitted with a modified version of the engine. This new modified engine takes a 3x3 fuel matrix, which, surprisingly enough, also has to take fuel in ratios that form a magic square.

Since you wrote the control program for the original engine, you've been drafted in to update your program to work with the new engine.

Adjust your program so that it supports generating magic squares of any size.

My solution to this puzzle is here - the password to the archive will be released in the next post.

Find the last post here: Binary Biomass

Picking the right interface for multicast communications

A network cabinet

At the recent hardware meetup, I was faced with an interesting problem: I was trying to communicate with my Wemos over multicast UDP in order to get it to automatically discover my PixelHub Server, but the multicast pings were going out over the wrong interface - I had both an ethernet cable plugged in and a WiFi hotspot running on my integrated wireless card.

The solution to this is not as simple you might think - you have to not only pick the right interface, but also the right version of the IP protocol. You also have to have some way to picking the correct interface in the first place.

Let's say you have a big beefy PC with a wireless card and 2 ethernet ports that are (for some magical reason) all in use at the same time, and you want to communicate with another device over your wireless card and not either of your ethernet ports.

I developed this on my linux laptop, but it should work just fine on other OSes.

To start, it's probably a good idea to list all of our network interfaces:

using System.Net.NetworkInformation;

// ...

NetworkInterface[] nics = NetworkInterface.GetAllNetworkInterfaces();
foreach (NetworkInterface nic in nics)
{
    Console.WriteLine("Id: {0} - Description: {1}", nic.Id, nic.Description);
}

This (on my machine at least!) outputs something like this:

eth0 - eth0
lo - lo
wlan0 - wlan0

Your machine will probably output something different. Next, since you can't normally address this list of network interfaces directly by name, we need to write a method to do it for us:

public static NetworkInterface GetNetworkIndexByName4(string targetInterfaceName)
{
    NetworkInterface[] nics = NetworkInterface.GetAllNetworkInterfaces();
    foreach (NetworkInterface nic in nics)
    {
        if (nic.Id == targetInterfaceName)
            return nic;
    }
    throw new Exception($"Error: Can't find network interface with the name {targetInterfaceName}.");
}

Pretty simple, right? We're not out the woods yet though - next we need to tell our UdpClient to talk on a specific network interface. Speaking of which, let's set up that UdpClient so that we can use it to do stuff with multicast:

using System.Net;

// ...

UdpClient client = new UdpClient(5050);
client.JoinMulticastGroup(IPAddress.Parse("239.62.148.30"));

With that out of the way, we can now deal with telling the UcpClient which network interface it should be talking on. This is actually quite tricky, since the UdpClient doesn't take a NetworkInterface directly. Let's define another helper method:

public static int GetIPv4Index(this NetworkInterface nic)
{
    IPInterfaceProperties ipProps = nic.GetIPProperties();
    IPv4InterfaceProperties ip4Props = ipProps.GetIPv4Properties();
    return ip4Props.Index;
}

The above extension method gets the index of the IPv4 interface of the network interface. Since at the moment we are in the middle of a (frustratingly slow) transition from IPv4 and IPv6, each network interface must have both an IPv4 interface, for talking to other IPv4 hosts, and an IPv6 interface for talking to IPv6 hosts. In this example I'm using IPv4, since the Wemos I want to talk to doesn't support IPv6 :-(

Now that we have a way to get the index of a network interface, we need to translate it into something that the UdpClient understands:

int interfaceIndex = (int)IPAddress.HostToNetworkOrder(NetTools.GetNetworkIndexByName4(NetworkInterfaceName));

That complicated! Thankfully, we don't need to pick it apart completely - it just works :-)

Now that we have the interface index in the right format, all we have to do is tell the UdpClient about it. Again, this is also slightly overcomplicated:

beacon.Client.SetSocketOption(
    SocketOptionLevel.IP,
    SocketOptionName.MulticastInterface,
    interfaceIndex
);

Make sure that you put this call before you join the multicast group. With that done, your UdpClient should finally be talking on the right interface!

Whew! That was quite the rabbit hole (I sent my regards to the rabbit :P). If you have any issues with getting it to work, I'm happy to help - just post a comment down below.

A 24 port patch panel.

Sources

  • Stackoverflow answers: 1, 2
  • Openclipart images: 1, 2
Art by Mythdael