Google command line tools: Google calendar task scheduler

The other day I stumbled upon something called googlecl which packs different clients for different google services. I couldn't wait to test it out so in combination with an idea from the work of this guy, I decided to write a task scheduler entierly controlled by google calendar.

Since I need training in bash scripting, plus it is kind of a simple thing to do - I decided to write it as a bash script.

It is dead simple to schedule a task. It can be run once or periodically.

Sorry for the image being in swedish, the language settings did affect gmail but not google calendar. Anyway, the procedure is as follows.

1. Install the googlecl tools. Use it with your calendar so that you are forced to complete the verification procedure. (i.e $ google calendar today).
2. Fire up google calendar and create a new task.
3. Enter a title for the task. The title must begin with gcmd for the scheduler daemon to pick it up.
4. Enter the start and finish times (currently the time range cannot span over several days).
5. In the "where" field, start with the flag -i followed by the interval with which to run the task (in seconds, 0 to only run it once).
6. Continue by adding -- which means "end of arguments".
7. Finish up by entering your command of choice last. Note that the initial run is scheduled using at, which does not have any PATH variable set. In other words, give full path names.
8. Run the gcmd script.

It is probably best to schedule the gcmd script to run every minute or so using crontab to have it catch the updates when you're adding tasks at work, the bus, the car, the empire state building, you get the point.

Let's look at the script. It's quite commented so I don't feel a need to explain anything further.

gcmd

#!/usr/bin/env bash

# 1. Get todays tasks from google, grab only those starting with gcmd.
#    Loop over the different tasks.
google calendar today --fields "title,where,when,id" \
  | awk -F, '/gcmd/' \
  | while read line; do

  # 2. Default args. These are overridden by args read from the
  #    where field of the task.

  # Interval between executions, 0 to only run once.
  interval=0

  # 3. Get arguments from the where field and overwrite default values.
  set -- $(getopt i: "$(echo "$line" | awk -F, '{print $2}')")
  while [ $# -gt 0 ]; do
    case "$1" in
      (-i) interval=$2; shift;;
      (--) shift; break;; # Terminates the args
      (-*) echo "$0 error - unrecognized arg $1" 1>&2; exit 1;;
      (*) break;;
    esac
    shift
  done

  # 4. Get the unique ID and construct a filename of it. The id is displayed
  #    as a URL, ending with a unique id.
  arr=($(echo "$line"|awk -F, '{print $4}'|tr "/" " "))
  filename="/tmp/.gcmd_${arr[${#arr[@]} - 1]}"
  command="$*"

  # 5. If the job has already been scheduled, un-schedule the job and
  #    remove the file
  if [ -e "$filename" ]; then
    echo "File exists, re-scheduling"
    atrm $(cat "$filename")
  else
    echo "New job \"$(echo ${line}|awk -F, '{print $1}')\""
  fi

  # 6. Get the desired time to run/stop. This should be improved to account for
  #    date aswell. Currently we only parse the time, which isn't enough if we need
  #    to run the task for several days.
  time="$(echo ${line}|awk -F, '{print $3}'|awk -F- '{print $1}'|awk '{print $3}')"
  stoptime="$(echo ${line}|awk -F, '{print $3}'|awk -F- '{print $2}'|awk '{print $3}')"


  # 7. If an interval is specified, make the command run according to the interval specified.
  #    This can be improved since it doesn't take into account how long the job takes to run so
  #    the interval will be more like interval + time to run task.
  if [ $interval -ne 0 ]; then
    command="stoptime=$(date -d ${stoptime} +%s); gcmdfun() { ${command}; }; while [ \$(date +%s) -lt \$stoptime ]; do gcmdfun; sleep $interval; done;"
  fi

  # 8. Schedule the command to run using at and store the job number in a file (named by the unique id).
  echo "$command" | at "$time" 2>&1 | awk '/job/{print $2}' > "$filename"
done

It's late over here now. Enjoy and Good night!

Cpulimit: take control over your CPU

Cpulimit is a tool that is very handy to have when doing data processing and other time consuming tasks. What it does is that it limits the CPU utilization of a process/program to a certain amount.

Imagine you started a cpu-hogging process and needs resources for other things

$ cpulimit --limit=50 --pid=1234


That will attach to the process and limit it's usage of cpu cycles to 50 percent of the available.


Cool!

5 Linux shell commands you should know about

5 Linux shell commands you should know about

Over the years I have gotten used to some very nifty commandline tools that I use more or less every day. Let's go through five of them right here.

This article will assume you have basic knowledge of the bash shell (good tutorial here http://mywiki.wooledge.org/BashGuide).

#1 tee

The `tee' command copies standard input to standard output and also to any files given as arguments.  This is useful when you want not only to send some data down a pipe, but also to save a copy. 

$ ./longrunningprocess | tee data.log

That is probably the simplest way you can use tee. Since it passes the stream on to stdout again you can use it as a proxy - storing the data to file but passing it on to the next command.

$ cat data | awk '{print $1+1}' | tee plusone | awk '{print $1-2}' > minusone

What that line does is - take the file data, containing some rows with numbers in it, pass it to awk and add one to each line, pass the modified stream to tee which stores it to file and once again passes it on to awk which subtracts two and writes it to another file.

Another more interesting way to do the same thing is

$ cat data | tee >(awk '{print $1+1}' > plusone) >(awk '{print $1-1}' > minusone)

which sends the output of data to two pipes running their separate versions of the awk script.

#2 wget

Wget is one of those tools that I use the most. Combining it with tee can make it a powerful tool.

$ wget www.kernel.org/pub/linux/kernel/v3.0/testing/linux-3.4-rc4.tar.bz2 -O - | tee kernel.tar.bz2 | tar xjvf -

That will download the linux kernel, have tee store it to file while tar decompresses it on the fly.

#3 awk

GNU awk is totally invaluable to me. I won't go in-depth on it here but look at my earlier post covering awk http://simonslinuxworld.blogspot.se/2012/04/awk-tutorial-by-example.html

#4 sed

Sed is a stream editor - that is - pass a stream to it, tell it what to edit and how to do it - store the output, or do something equally useful with it.

Let's say I know a guy, who knows a guy who once downloaded a season of a TV-show illegally. He told me how he needed to rename the files of the show to a specific naming convention for his XBMC media center to be able to download information about the show from some website. In this case (and many others) sed is needed.

$ ls | sed -r "s/(.+)_(.+).mkv/mv & \"Series (2012) \2.mkv\"/" | bash

Piece of cake - list files, substitute the filename series_S01EXX.mkv  for mv series_S01EXX.mkv "Series (2012) S01EXX.mkv", pass it to bash for evaluation.

If you want bittorrent to still find the original files just make symlinks instead of moving the file (that's what he did).

#5 xargs

xargs reads items from stdin, delimited by blanks or newlines, and executes the  command one or more times with any initial arguments followed by items read from standard input. Blank lines on the standard input are ignored.

Example: Remove all files matching pattern *~ (tempoary emacs files) recursively.

$ find . -name "*~" -type f -print | xargs rm -f


If you need more control over how the items are inserted into the command to execute you can use {}, which gets substituted by the actual item when the command is executed. E.g.

$ find . -name "*~" -type f -print | xargs -n 1 -I{} mv {} /tmp


Since mv only accepts two paths we add -n 1 to xargs to make it execute each command with one of the arguments. -I{} is used to tell xargs to use {} for substitution.

Final words

I hope you have enjoyed this little infomative post about useful linux commands. There's a lot of options to them so I suggest you check out the manpages to make full use of the commands.