Syslog to MySQL

I had a need to put Syslog data into a database for further analysis.
Using one of my linux boxes this turned out to be much easier than I thought, at least for basic functionality.

Basically I followed the instructions at http://www.rsyslog.com/doc/rsyslog_mysql.html
 and everything just worked.

I already had MySQL installed and Rsyslog was the default syslog on my Linux distribution so I used the included script to create the database, then I created the account.

mysql -u root -p < /usr/share/doc/rsyslog-mysql/createDB.sql

grant all privileges on Syslog.* to 'Syslog'@'%' identified by 'password' with grant option;

Then I modified the rsyslog config file.

#put this line as the first line in /etc/rsyslog.conf
$ModLoad ommysql

#this line logs everything
*.*       :ommysql:127.0.0.1,Syslog,Syslog,password
#*.*       :ommysql:127.0.0.1,db-name,db-user,db-password

Then I restarted the Rsyslog process.
/etc/rc.d/init.d/rsyslogd restart

Then I issued a 'SELECT * FROM Syslog.SystemEvents;'


I saw that the local machine was logging all of its syslog data and the other devices that were logging to the syslog server were also showing up in the database. As a point of interest, all of the data was still being put in the regular syslog file as well.

Future work on this will be to tailor what gets logged and some possible customizations.

Create an Animated Gif with ImageMagick

I needed to create an animated series of images to observe some change over time.
To do this I decided to use ImageMagick on my linux machine. I save all of the images in a folder named image1.png, image2.png, image3.png, ... .

Then I used the following command.

convert -verbose -delay 200 -loop 0 image*.png output.gif

This takes the files named image*.png in the order that they are listed and makes a GIF out of them with a 200 hundreths of a second delay between each frame. The -loop 0 flag makes it loop indefinitely. You can put 4 in there to loop 4 times etc.

This worked fine but the order was not correct when I got over 10 images so for now I renamed them to imageA.png and went from there.

That solved my immediate need for an animated image but I need to find a neater solution to sorting the filenames.

Small Business Server 2008 Router Port Mapping

Recently my Small Business Server 2008 machine started complaining about not being able to open the router ports. I think it happened when I ran the Fix My Network Wizard to update the expiring certificates.

At some point in the past year I must have enabled uPnP on the router interface. When the wizard detects the uPnP it tries to configure the router and since it could not it was giving me error messages.

I spent a few hours trying to find some documentation on the wizard and/or figure it out. ONe of the closest places with good info was http://sbs.seandaniel.com/2008/10/fix-my-network-wizard-in-sbs-2008.html.

In the end I just turned off uPnP and reran the wizard and the errors went away.

Firefox Out of Process Config

I have run into a problem with Firefox when viewing a pdf. The pdf would not finish downloading or take a long time and it would lock up the entire browser. I couldn't switch to another tab or do anything. For some reason this seemed to happen very often when reading academic papers found on Google Scholar.

Since 3.6.4 Firefox has had the option of having a plugin run in another process. It is part of the Electrolysis project. By default  QuickTime, Flash, and Silverlight on their own process, but you can also manually add other plugins via about:config.

I wanted to add Adobe reader to see if this would help and this is how I did it.

Open about:config

right click and create a new boolean preference

name it dom.ipc.plugins.enabled.nppdf32.dll

set the boolean value to True


Restart the browser for the settings to take effect. This setting can be reversed by changing the boolean value to False.


For other plugins you can create by adding the plugin name. Those name can be found in about:plugin.

Turn of Snap in Windows 7

I am not crazy about the Snap feature in Windows 7.
The Snap feature allows you to arrange open windows, including maximizing and resizing, just by dragging and dropping a window to different edges of the screen.

To disable it click on the start button and type snap in the start search box.
A window that looks like this will open.

Click on "Prevent Windows from being ...." and then ok.

Now you should be able to move the windows and place them where you like.

Parsing Atom Feeds with Python

One of the things that I have really come to like about Python is how easy it is to prototype some code to try an idea out. I wanted to parse some blog feeds to try out some semantic analysis algorithms.

I just imported feedparser and let 'er rip.
Here is a simple example to print out the titles of this blogs Atom feed.

import feedparser

def parseit( feedurl ):
    
    data = feedparser.parse( feedurl )

    print 'Title => ' + data['title']

    print 'There are %d entries.' % (len(data['entries']))

    for entry in data['entries']:
        print entry.title

if __name__ == "__main__":

    print 'Running'
    parseit('http://dataoracle.blogspot.com/feeds/posts/default')

Dynamic R Plotting

I needed to create a graph that contained multiple lines of data.
This is relatively easy to do in R when you know about the data like the maximum values and the number of lines.
What is not so easy is to create an R script that will dynamically create the graph without knowing anything about the data ahead of time.

It appears that the call to plot() needs to have sort of parameters to create graph.
Just calling plot.new() isn't going to give me the results that I need.
I am certain that there is a better way to do this but it works.
I just used a counter in the loop and if it is the first time through I call plot with the first line.
If it is not the first time through then the lines() method is called to add the data.

The colors are dynamically created based on the number of lines.

# required libraries
library(RMySQL)

cntr <- 1# set a counter that will represent the # of times through the loop

# connect to db using credentials in my.cnf
dbcon <- dbConnect( MySQL(), group="localnfl")

# get the years with data from the database
# returns a data frame
years <- dbGetQuery( dbcon, "SELECT DISTINCT( year) FROM nfl.bet_lines")

#define the image that will hold the graph
png("../../public_html/betline.png", width=1024, height=1024)

#create a vector to hold text for legend
legend_text <- character()

# choose enough colors (one for each year)
# returns a character vector
colors = rainbow( length(years$year) )
# iterate over each year
for( y in years$year )
{
  #create a string that will be the SQL
  # used this as a poor man's prepared statement to pass in year
  sql <- sprintf("SELECT IF( (hscore-vscore)>0 AND lineopen>0,
                         ABS(lineopen- (hscore-vscore))*1,
                         ABS(lineopen- (hscore-vscore))*-1) AS mov
                  FROM nfl.bet_lines
                  WHERE week <= 17 AND year = %d
                  ORDER BY mov asc",y )

  # run the query returns a one column data frame
  values <- dbGetQuery( dbcon, sql )

  # if this is the first time through create the plot with
  # type="n" for no actual plot ( could probably just plot the first line)
  if( cntr == 1 )
  {
    # create the plot with no axes labels( ann =FALSE )
    plot( row.names(values), values$mov, ylim=c(-50,50), type="n", ann=FALSE )
  }

  lines( values$mov, type="l", lwd="2", col=colors[cntr] )

  #add the current year numeric value to the legend text as a string
  legend_text[length(legend_text)+1] = as.character(y)

  cntr <- cntr + 1  #increment the counter
}

#create the main title, double font size(cex=2), italic is 4
title(main="Mad Title", cex.main=2, font.main=4)

# place text on top(3)
mtext( "actual > 0 and opening line > 0", 3 )

# Label the x and y axes
title(ylab="| Line - Actual |")
title(xlab="Game Number")

#create the legend
legend( "topleft", legend_text, fill=colors )

dev.off()

dbDisconnect(dbcon) 

Let me know if you have any suggestions on improving this R script.