Using a task manager to create blog posts

I’ve been using the static site generator PieCrust for several years now to manage this site and its blog. It’s been really nice having my site in plain, raw HTML files with the bonus of being able to write it all up in Markdown. There’s always been a downside, though: the mechanics of publishing a new post. It’s a process involving rsync, git commits, ssh, or a mixture of these. Contrast this to something like Wordpress with its slick web administration/editor and it feels much more cumbersome than it should be. I’ve always been in search of something that allowed me to start writing blog posts as quickly and easily as possible.

A few months back I stumbled upon myTinyTodo, a very quick and simple web-based task manager. It’s a pretty straight forward app in terms of task management: there’s a field where you type in a task you need to do, push "enter" and it’s added to a list. However, there were a few more features that made it particularly interesting. First, a time stamp is affixed to the task upon creation. Second, it’s the task’s title that you’re writing in the field when you first add a task to the list. When you edit the task, a second field is presented for the addition of expanded content as well as a third field for tags to be associated with the task. Third, an RSS feed can be enabled for each to do list (you can have multiple to do lists, each on its own tab within the app). When you combine these features, you have everything you need to publish a blog. If you wanted to skip any HTML presentation of a blog, just hand out the RSS link for the todo list and you have an instant blogging engine right next to your to do lists. I took this a step further, though, so that I could incorporate it into PieCrust.

  1. I wrote a PHP script using SimplePie to pull in the RSS feed. This script does a couple of things:
    • Checks to see if each task in the feed has already been added as a post to PieCrust. If it has, it will make a diff between the content it’s pulling in and the content that already exists in PieCrust; merge/create as needed.
    • Inserts the post’s metadata section at the top of the post’s page (things like author, tags, date published, template to use, etc)
  2. The script is run via cron every minute. I’ll probably write a custom daemon at some point so that I don’t have to wait a whole minute for cron to run.
  3. I wrote a custom daemon using inotify so that PieCrust can check to see if anything has changed in its posts directory. In other words, any time the script from step one adds a new post or edits an existing post, inotify will see the change to the "posts" directory and kick off a PieCrust update.

After thinking and searching for awhile, I believe I’ve finally found for me the perfect marriage of simple and portable content presentation with PieCrust and the ease and convenience of web-based content creation with myTinyTodo.

This post was created with myTinyTodo and PieCrust

Execute a different command on the previous file

I often want edit a script with VI and then run it. There’s probably other ways to do this, but this is working for me:

# open file in VI
vi /usr/local/bin/my_script.php

# make some changes

# save the changes
esc :w!

# put VI in the background
^z

# use bash's previous command shortcut with :1 to represent the first argument of the previous command
php !!:1

# see results

# resume VI for more editing
fg

References

Bash history and bang commands

Create Asterisk Recordings on OS X

My wife and I used to record our IVRs ourselves using a mic and Audacity. As I’ve needed to change my IVRs, I’ve found it easier to just write up a text file and feed it into say. The “trick” was to 1) insert long enough pauses in the input file (you can do this by placing one or more semicolons where you want a pause) and 2) get all of the say flags correct (say’s man page helped sort this out).

ivr.txt

Note the semicolons. Add as many as you want for longer pauses.

dial one for something;;
dial two for something else;;
dial three for yet another thing

say

the command with its flags:

say \
  --voice=Victoria \
  --file-format=WAVE \
  --data-format=LEI16@8000 \
  --input-file=ivr.txt \
  --output-file=ivr.wav

The resulting audio file is something you can upload and plug into Asterisk without need for further conversion. The say command will run very quickly; so quickly that I thought it didn’t work. However, I was able to open it in VLC and verify the “Codec Details” were correct from “Media Information” window.

Apache Access Log Summary

Log parsers and traffic analyzers like awstats and piwik are great, but sometimes I want a really quick way to see what pages were visited and by whom in a given day. Email is convenient for me and as I review a number of logs there every morning, I thought having this Apache summary there as well would be nice. Use as you wish and let me know if you have any suggestions for improvement.

if [[ -z $1 || -z $2 ]]
then
cat << EOF

A simple script to display, or optionally email, a breif output of the pages visited in a given day.

usage: $0 [logfile] [datestring] <1>

* Can only display as far back as the current log file for the given host. i.e. it won't pull data from rolled over log files
* Works with standard Apache log files

OPTIONS:

  [logfile] Required. The full path of the http access log file that you want to review
  [datestring]  Required. e.g. 2013-12-17 or \$(date --date="1 day ago" +\%Y-\%m-\%d) for a nightly cronjob
  <email>   Optional. Enter an email address to produce an email instead of displaying to standard out. Useful for a nightly cronjob

EOF
exit
fi

function output() {
grep $(date --date="$2" +%d/%b/%Y) $1 \
  | grep -v "bot\|feedafever.com\|css\|Ezooms\|\/js\/\|favicon.png\|apple-touch-icon.png\|\ 301\ \|\ 302\ " \
  | awk -F " " '{print $4 " " $1 " " $7}' \
  | sed 's/\[//' \
  | sed 's/google.*/google.com\"/' \
  | sed 's/bing.*/bing.com\"/' \
  | sed 's/https...\|http...//' \
  | uniq
}

if [[ -z $3 ]]
then
  output $1 $2
else
  output $1 $2 | mail -s "$(basename $1)" $3
fi

Call Detail Report Summary

I find the CDR in FreePBX to be noisy, especially when there are ring groups and the like. As such, finding what I’m interested in from the CDR can be more of a chore than it should be. The CDR in Switchvox is pretty sweet, but, alas, I don’t have one. I think the FreePBX team is working on a more elegant solution; not sure if it will be included in the distro or be a commercial addon. In any event, I wrote up the little script below. It pulls data from asteriskcdrdb.cdr and shows only the info in which I’m interested. Use as you wish and let me know if you have any thoughts for improvement.

if [[ -z $1 ]]
then
cat << EOF

A simple script to display, or optionally email, a brief summary of a given day's CDR.

usage: $0 [datestring] <email>

OPTIONS:

  [datestring]  Required. e.g. 2013-12-17 or \$(date --date="1 day ago" +\%Y-\%m-\%d) for a nightly cronjob
  <email>       Optional. Enter an email address to produce an email instead of displaying to standard out. Useful for a nightly cronjob

EOF
exit
fi

function output() {
  echo "SELECT t.calldate,t.clid,t.dst,t.duration \
  FROM ( \
    SELECT * FROM asteriskcdrdb.cdr \
    WHERE calldate LIKE '$1%' AND lastdata NOT LIKE '%auto-confir%' AND dst!='pbdirectory' AND channel LIKE 'SIP%' AND dst NOT LIKE '*%' \
    ORDER BY duration DESC \
  ) \
  t \
  GROUP BY t.calldate \G" \
  | mysql \
  | sed 's/vmu/Voicemail - /;s/dst: s/dst: hang/;s/dst/destination/;s/clid/source/;s/hang/hang up/'
} 

if [[ -z $2 ]]
then
  output $1
else
  output $1 | mail -s "Call Detail Report" $2
fi

Porting a Phone Number to TracFone

My Wife and I are looking to cut our cell phone bills by moving over to TracFone. We’ve been happy with the service from our current provider, but the pricing is quite high in today’s market. Below are my notes and summarization of the porting process as discribed on TracFone’s FAQ. Please refere to TracFone’s documentation for more information and official support.

Porting Requirements

  1. The phone number must be eligible to be ported
    • Eligibility will be checked at the initiation of the porting process
    • Pager numbers, special use numbers, and toll free numbers are not eligible for porting
  2. The phone number that you wish to transfer is still active with your carrier
    • Deactivation will happen automatically through the transfer process
  3. The phone number is being ported within a local service area
    • e.g. You can transfer a number within Atlanta, but not from Los Angeles to Atlanta
  4. You have a TracFone wireless phone
  5. Your TracFone number is active in TracFone’s system
  6. Your information on the Transfer Out request sent to TracFone from the other carrier matches the information in TracFone’s system.

Information Required to Initiate a Port Request

  1. Your name
  2. Your address
  3. A phone number
    • Must be a number to be reached at other than number requested for porting
  4. Your Social Security Number
  5. A recent bill with your current service provider
    • Required to ensure TracFone has the appropriate billing information when they send the transfer request to your current provider

How to Keep your Phone Number After it has been Successfully Ported

  1. You must purchase and add a TracFone Prepaid Wireless Airtime card before the Service End Date that is displayed on the TracFone.
    • If you do not purchase and add more airtime prior to the Service End Date, your TracFone Service will deactivated and you will lose your phone number.
    • If you lose your phone number you will not be able to reactivate the wireless number with any other wireless or wire line service provider.
    • A deactivated TracFone can be reactivated and a new phone number will be assigned after you purchase and add any denomination TracFone Prepaid Wireless Airtime card.
    • The addition of more TracFone Prepaid Wireless Airtime cards extends your active service 30, 90, or 365 days from the date the card is added depending on the airtime card
    • Your Service End Date will be displayed every time you turn on your TracFone.
  2. If your TracFone service remains inactive over 30 days, you will lose your telephone number.

Additional Terms and Conditions

  1. Your request to port-in your telephone number must be approved by your Current Service Provider before TracFone can activate service with your requested telephone number.
  2. Once approved, your TracFone wireless phone will be programmed with your requested number.
  3. The transfer process typically takes a few hours to complete but could take as long as 2 business days or longer for landlines.
  4. Until the porting process is completed, you will not be able to make a call other than calls to 911.
  5. If you need to call 911 please notify the 911 operator of your location since they will not be able to call your phone number during the porting process.
    • If you are in an area where there is no signal, it is highly probable that a call to 911 will not go through.
    • Do not rely on this function in an emergency situation. Locate the nearest landline phone and call for help.

UPDATE 1: My wife’s phone number has finished porting. We started the port process with TracFone ~noon on a Saturday and it took ~24 hours. Both my wife’s cell number and my cell number were associated with a single account at our old provider. We were more concerned with getting my wife’s number ported since more people call her directly. As such we started with hers. I was afraid the second number would just vanish after the first one was ported; I don’t mind if mine vanishes. Once my wife’s was ported, I went ahead and kicked off a port of mine. Let’s see if it’ll work.

UPDATE 2: Wow, that was fast. By the time I finished UPDATE 1 above and immediately checked my TracFone account, the port was finished! Turn around time was less than half an hour.

Use DNS to Find All IPs and Hostnames

My company includes the user’s name as one part of a workstation’s complete hostname. From time to time I need to lookup the IP or complete hostname for that user. There are several ways to do this, but I keep the below one-liners in ClipMenu. Having them within easy reach makes them a convenient method to get the required info, even if I’m SSHed into a remote network with otherwise limited access. As written, they work on a standard Class C network, scanning everything from .1 to .254. Change the net variable to match the network you’re scanning.

Resolve IPs

net=10.1.2;i=1; while [ $i -lt 255 ]; do line=$(nslookup $net.$i|grep name|awk -F" " '{ print $1 " " $4 }'); name=$(echo $line| awk '{ print $2 }');ip=$(echo $line| awk '{ print $1 }'|awk -F. '{ print $4 "." $3 "." $2 "." $1 }'); echo $name $ip; ((i++)); done | grep -v "\.\.\." | sort

Resolve Hostnames

net=10.1.2;i=1; while [ $i -lt 255 ]; do line=$(nslookup $net.$i|grep name|awk -F" " '{ print $1 " " $4 }'); name=$(echo $line| awk '{ print $2 }');ip=$(echo $line| awk '{ print $1 }'|awk -F. '{ print $4 "." $3 "." $2 "." $1 }'); echo $ip $name; ((i++)); done | grep -v "\.\.\."

User Profile SSH Config File

Background

I hadn’t used a ~/.ssh/config since I’m such a curmudgeon. However, as I’m managing hosts on more and diverse networks from a single machine, I needed a way to make everything gel.

  1. The first of these networks require kerberized logins. There’s a fallback password prompt, but I always want to make use of my TGT. This Kerberos Identity is also not my default identity.
  2. The second network uses key-based authentication. To prevent this from conflicting with my default key, I set the key to be used. The saves me from having to use an -i flag.
  3. I also have a machine in EC2 that has a different key.

Solution

Host <alias>
    hostname <host.domain.tld>
    user <username>
    GSSAPIClientIdentity <username@DOMAIN.TLD>
    PasswordAuthentication no

Host <alias>
    hostname <host.domain.tld>
    IdentityFile ~/.ssh/<id_file>
    port <port number>

Host <alias>
    hostname <host.domain.tld>
    user <username>
    IdentityFile ~/.ssh/<other_id_file>

Now when I go to ssh, I just type

ssh <alias>

and I’m in. No more flags!

References

Fortigate Traffic Capture

The below can be used to verify pings are going out the correct policy routes on a Fortigate

diagnose sniffer packet any 'dst host 8.8.8.8 and icmp or dst host 8.8.4.4 and icmp'

References

Adding Tipue Search to a Piecrust Blog

Background

One thing I’ve missed since my migration away from Wordpress is the lack of search. My workaround for quickly finding a particular post was to load up my archive page and do a command + f which would pull up any hits on titles and tags. This worked well enough for me, on this still growing site. However, I have other sites where an integrated search option would be very handy, especially if I concern myself with my visitors’ expectations.

Enter onethingwell.org’s fantastic curation of apps. It was OTW that first turned me on to Piecrust and it again came to the rescue with its post on Tipue Search. Tipue Search “is a site search engine jQuery plugin” that “only needs a browser that supports jQuery”. It doesn’t need a database on the backend or a large PHP, Python, Ruby, Java app powering it. Those kind of solutions are fine for much larger and dynamic sites. However, for sites like this one, they are overkill and add unnecessary overhead with no appreciable return; they’re not the right tool for the job.

Start Cooking

Tipue Search has three modes:

My first attempt at integrating Tipue Search with my site was by way of the “live” mode. Afterall, if I can just sit back and let it take care of all the intelligence, then all the better. As it turned out, though, live mode is not dynamic; you can’t point it at a directory and let it crawl the pages on demand. This makes sense as such a method would take longer and longer to process as a site grew. It’s fine for the type of site that has a few pages. When using it on a blog site like this one, the best I could do was to get it to search my Archive page. It would always return a hit, but that hit was the entire archive page. It wouldn’t point me to the specific post that it found. It was as good as having no search.

After I ran into that wall, I decided to just have Piecrust build a JSON file that I could point Tipue Search to with its JSON mode. This is exactly what static site generators are built to do after all.

The Solution

  1. Add the requisite Tipue Search scripts, stylesheets, and form to your default template.
  2. Create a new template, searchdata.json:

    {"title": "{{ post.title }}", "text": "{{ post.content|replace({'\n': " ", '\\': " "}) }}", "tags": "{% if post.tags %}{% for tag in post.tags %}{{ tag }}{% if not loop.last %}, {% endif %}{% endfor %}{% endif %}", "loc": "{{ post.url }}"},
    
  3. Create a new page, searchdata.html, that loops through each blog post using the above template for the output.

    ---
    content_type: json
    layout: none
    posts_per_page: 9999
    single_page: true
    format: none
    posts_filters:
        not:
            has_tags: draft
    ---
    {"pages": [
        {% for post in pagination.posts %}
        {% include 'searchdata.json' %}
        {% endfor %}
        {"title": "", "text": "", "tags": "", "loc": ""}
    ]}
    
    • The null value line just after the for loop is needed to create a valid JSON file that doesn’t have a comma for the final definition line.
  4. Finally, create your search.html page which will be the landing page for your search results. Be sure to point it at the generated searchdata.json file.

    <div id="tipue_search_content">
      <div id="tipue_search_loading">
      </div>
    </div>
    
    <script>
    $(document).ready(function() {
        $('#tipue_search_input').tipuesearch({
          'mode': 'json',
          'show': 9999,
          'showURL': false,
          'contentLocation': '{{ site.root }}searchdata.json'
        });
    });
    </script>
    

Caveats