• Sending Commands to Multiple EC2 Instances Using dsh

    We are moving Adafruit IO’s node.js services to a group of t2.nano EC2 instances, and I wanted to document my dsh setup. I used to use it for managing the backend workers when working on data.sparkfun.com. It came in really handy when I needed to quickly check or kick anything on the servers.

    First, you will need to install dsh using brew, apt-get, yum, etc:

    $ brew install dsh
    

    You will need to make a couple config directories for your user:

    $ mkdir -p ~/.dsh/group
    

    Create a new dsh.conf file using your favorite text editor:

    $ vim ~/.dsh/dsh.conf
    

    Paste the following config into ~/.dsh/dsh.conf, and save the file:

    showmachinenames = 1
    remoteshell = ssh
    

    Now, you can create a dsh group for the set of servers you would like to send commands to. Create a new file called example in the ~/.dsh/group folder using your favorite text editor:

    $ vim ~/.dsh/group/example
    

    Enter the host names for your servers, and save the file:

    example-1.uniontownlabs.org
    example-2.uniontownlabs.org
    example-3.uniontownlabs.org
    example-4.uniontownlabs.org
    

    Now you can send commands (like uptime) to all servers in the group using the following command:

    $ dsh -g example -w 'uptime'
    example-1.uniontownlabs.org:  14:17:44 up 17:15,  0 users,  load average: 0.08, 0.03, 0.05
    example-2.uniontownlabs.org:  14:17:43 up 16:36,  0 users,  load average: 0.00, 0.01, 0.05
    example-3.uniontownlabs.org:  14:17:45 up 16:41,  0 users,  load average: 0.00, 0.01, 0.05
    example-4.uniontownlabs.org:  14:17:45 up 16:41,  0 users,  load average: 0.00, 0.01, 0.05
    

    Using -w will send the command to each server in the order they appear in the group config, and will wait for a response before continuing. If you want to send the commands concurrently to all servers in the list, use the -c option to send the command:

    $ dsh -g example -c 'uptime'
    example-1.uniontownlabs.org:  14:22:42 up 17:20,  0 users,  load average: 0.00, 0.01, 0.05
    example-3.uniontownlabs.org:  14:22:42 up 16:46,  0 users,  load average: 0.00, 0.01, 0.05
    example-2.uniontownlabs.org:  14:22:41 up 16:41,  0 users,  load average: 0.00, 0.01, 0.05
    example-4.uniontownlabs.org:  14:22:42 up 16:46,  0 users,  load average: 0.10, 0.05, 0.05
    

    You can get an interactive shell for all servers in the group by using the -c and -i commands together. Use CTRL-D to exit the session:

    $ dsh -g -c -i
    
    read more →
  • Caching YouTube Videos

    Here are some quick instructions if you would like to grab high resolution copies of all of the videos in a YouTube playlist.

    Install dependencies (replace brew with apt-get or yum for linux):

    $ brew install youtube-dl ffmpeg
    

    You can get a muxed 1080P version of all videos in a playlist by running the following command and replacing the URL:

    $ youtube-dl -w --restrict-filenames -f 137+140 -o '~/%(playlist_title)s/%(upload_date)s.%(ext)s' https://www.youtube.com/playlist?list=PLjF7R1fz_OOU08_hRcayfVZSmTpBCGJbL
    

    The -w option will disable overwrites, and --restrict-filenames will make the saved filenames URL safe. The -f 137+140 option tells youtube-dl to grab the 1080P video and AAC audio files, and mux them using ffmpeg.

    read more →
  • Raspberry Pi dwc2 Gadget Notes

    Here are some notes from experiments with dwc2 otg enabled on a Raspberry Pi Zero.

    read more →
  • Message Queue Link Dump

    We use Redis fairly heavily for Adafruit IO, and I started wondering if there might be some better/faster options for our MQTT Redis message queue. It looks like there are many options that are much faster than Redis.

    read more →
  • Redis Killed by OS?

    Redis Server has been killed a few times this week by the OOM. More info in this post on stackoverflow.

    After setting the maxmemory limit to 3GB, and maxmemory-policy to allkeys-lru, things should have gotten better. For some reason node.js workers would die as soon as they were restarted, reporting that they couldn’t connect to Redis.

    Starting the process manually would connect successfully, so the problem seemed to be with pm2, which manages the node cluster.

    Fix

    I tried killing pm2 by running the following command.

    $ pm2 kill
    

    Then, I restarted the node cluster, and everything was back to normal.

    $ sudo service io-server-cluster restart
    
    read more →