SOLVED: Ubuntu installation of Canon MX452 Inkjet Printer

Unlike many printers and Linux, this install went simply.

• Unbox printer.
• remove orange packing tape.
• unbox power cords and usb cable, install.
• open front pull-down cover, and one under it - push down on grey loops gently and put in inkjet cartridges.
• Put paper in at bottom, only hold 50 sheets or so.
• turn on, wait.
• open terminal window, type sudo ls and enter pw.
• open browser, get download from http://support-sg.canon-asia.com/contents/SG/EN/0100515301.html
• in terminal, cd ~/Downloads
• tar -xzvf cnijfilter*
• cnijfilter*
• On printer, turn off and on again just in case.
• sudo ./install.sh
• follow prompts accepting defaults.
• in browser, open google.com and print open page as test. Should hear printer working.

Done.

Optimizing Python - getting data out of memcache with struct.unpack

So, I have this Memcache data store that holds timestamps and values from a monitoring application. Since each memcache key corresponds to an hour's data, I only need to store 2 bytes for the number of seconds past the hour. I don't care about duplicate data being stored, but on retrieval I'd like to eliminate it if it exists.

Input data is: (ts,val), (ts, val), ... encoded using Python's struct.pack command. The ts (timestamp) is (as noted) packed with format h (unsigned int). The val (value) is a floating point number of 4 bytes, packed with format f.

The original version of this encoding was:

    def OLD_rawDataToTsVals(self, timeOffset, raw):
        tsVals = []
        while raw:
            ts, val, raw = raw[:2], raw[2:6], raw[6:]
            ts = timeOffset + struct.unpack('h', ts)[0]
            val = struct.unpack('f', val)[0]
            tsVals.append((ts, val))
        return tsVals

I found this version:

• ran really slowly;
• didn't eliminate duplicate values;
• would really choke the longer the input data (as in 33,000 datapoints in an hour).

For the second version, I knew I had to stop with the copying of the data string over and over again, which I knew was eating major cycles.

Doing some math, I figured out I could iterate over the string, extracting each element and converting the two parts to python numbers.

    def rawDataToTsVals(self, timeOffset, raw):
        tsVals = []
        for i in range(0, len(raw), 6):
            rawtime = raw[i:i+2]
            ts = timeOffset + struct.unpack('h', rawtime)[0]
            val = struct.unpack('f', raw[i+2:i+6])[0]
            tsVals.append((ts, val))
        return tsVals

This was better timewise, but didn't remove duplicate data. I made the 'seen it yet' test occur even before the conversion to (int, float), which saved a bit of time doing useless conversions.

    def rawDataToTsVals(self, timeOffset, raw):
        tsVals = []
        seenTimes = set()
        for i in range(0, len(raw), 6):
            rawtime = raw[i:i+2]
            if rawtime in seenTimes:
                continue
            seenTimes.add(rawtime)
            ts = timeOffset + struct.unpack('h', rawtime)[0]
            val = struct.unpack('f', raw[i+2:i+6])[0]
            tsVals.append((ts, val))
        return tsVals

Yet, it was STILL TOO SLOW. Where was the time going? I timed the various parts and found the slow bit was the conversion to int/float. That unpack was happening a lot and the time added up.

I tried the following but FAILED.

        # BAD DON'T USE ** BAD DON'T USE **
        elems  = rawLen / 6.0  # 6 bytes per - 2=time + 4=data.
        intElems = int(elems)
        if (elems != intElems):
            self.log.warning("elems non-integer: len: %s" % (rawLen))
            return []
        unp = struct.unpack("hf"*intElems, raw)
        # BAD DON'T USE ** BAD DON'T USE **

The above fails because if we pack these things together, there's a word-alignment problem that unpack is unable to cope with. It would have to be something like (int, zeroes, float) to make the float align on a word boundary.

But, I couldn't give up, this had to work better. So, I extract all the ints, string those together and unpack them, then do the same thing with the floats.

HERE IS THE FINAL VERSION:

    def rawDataToTsVals(self, timeOffset, raw):
        tsVals = []
        seenTimes = set()
        try:
            rawLen = len(raw)
            times = ""
            vals  = ""
            for i in range(0, rawLen, 6):
                rawtime = raw[i:i+2]
                if rawtime in seenTimes:
                    continue
                times += rawtime
                vals  += raw[i+2:i+6]
            timesList = struct.unpack('h'*(len(times)/2), times)
            valsList  = struct.unpack('f'*(len(vals)/4),  vals)
            assert len(timesList) == len(valsList), "Lens of times and vals unequal, t=%s, v=%s" % (len(timesList), len(valsList))
            for i in range(0, len(timesList)):
                tsVals.append((timeOffset+timesList[i], valsList[i]))
            #self.log.debug("unpacked %d vals, len ts %s." % (len(timesList), len(tsVals)))
        except:
            tb = traceback.format_exc()
            self.log.debug("tb in rawDataToTsVals(): rawSize: %s, %s" % (rawLen, tb))
            pass
        
        if 0:  # debugging
            self.log.debug("tsvals: %s" % ( tsVals))
        return tsVals

Unpacking all the h's at the same time, and likewise the floats, makes everything align, and since it's one function call to struct, is very fast.

Enjoy!

Alternate Fortune Cookie Sayings

I had 'Panda Express' Chinese take-out for lunch, and found the fortune cookes to be too boring. Thus, I've created some of my own, in the hope they liven things up a bit. Feel free to reply with some, if you're inspired to do so.

• Avoid doing new things, you might get hurt.
• Your opinions are frequently wrong.
• Now is not a good time to invest.
• Your face betrays you.
• Other people work harder than you do.
• Your life has no meaning.
• Your lucky number is Zero.
• Avoid having opinions, you might be wrong.
• Do not finish any projects tomorrow.
• Avoid doing things that require too much thought.
• Your efforts are doomed to failure.
• Ask people for help, pleading stupidity.
• Insult the nationality of all new people.
• Your smile looks pathetic, only use it when begging.
• Your kids will always ignore you.
• Things are always as they seem.
• Bad news is coming, in large amounts.
• Fear your loved ones.
• Sunrises bode not well for your financial future.
• Steal everything before your friends stop liking you.
• Wear red on Tuesdays to avoid a dishonorable death.
• Cancel all your credit cards before it's too late.
• Your wishes of last Wednesday will never come true.
• Buy new underwear before they find out.
• Several former friends are conspiring against you.
• Never order fries with that.
• People's clothes indicate if they like you.
• Someone knows what you did that time.
• Invest only in companies starting with the letter F.
• Learn how to cook roadkill safely. Soon.
• To see far, sleep on your roof.
• Learn how to chop the head off an attacker.
• As of yesterday, praying became useless.
• The Antichrist was born last week.
• To kill the bugs, wash all your clothes on hot.
• Your spouse and your best friend have a secret.
• When you see the flash, duck, it might help.
• Horseback riding will be useless in 3 years.
• Farm animals like you until they smell you.
• Unplug your lamps and toaster before it's too late.
• A phone call will soon make you cry loudly.
• Fear everything, it's safer that way.
• Put your affairs in order before tomorrow night.
• Give away your posessions, but don't leave a note.
• Contemplate everything, but don't commit, ever.
• Physical pain can help you make hard decisions.
• To change the inside, change the inside.
• Lock up your women. 

Have any others?

Example of how to set JournalCommitInterval in MongoDB

I have found no way yet to find the current value of journalCommitInterval.

But, I have found how to set the journalCommitInterval.

We've got our mongodb set up to use a local disk for our journal, and data resides on a SAN-mounted mount point.

We'd like to set our journalCommitInterval up to a larger value to cut down on the IOPS to our local disk. This might have the added benefit of helping us run faster.

I've run the following command against a mongos, and it complains:

mongos> db.adminCommand({ "setParameter" : 1, "journalCommitInterval": 499});
{ "ok" : 0, "errmsg" : "journaling is off" }

When I run this against a specific daemon, it works up to a value of 499 (setting it to 500 generates a complaint/error):

shard-000:PRIMARY> db.adminCommand({ "setParameter" : 1, "journalCommitInterval": 499});
{ "ok" : 1 }

I have posed the follwoing questions to 10Gen:

• Question 1: do I need to run this against each mongod individually?
• Answer 1: Yes.
• Question 2: does this persist across restarts of the daemon?
• Answer 2: No. That daemon's journalCommitInterval returns to its default when a daemon are restarted. Thus, it's better to set it in the config file you have defined for the shard. Or, if you define it via command line, do something like:

  /path/to/mongod --journalCommitInterval=499 ...
  

I have just created a minor request in core-server to expose the current value so we can verify it has been set correctly on all shards. If you agree this would be a handy feature to have, please watch and VOTE on the case, https://jira.mongodb.org/browse/SERVER-10508

Solved: Installing Linux on Acer Aspire V5-122P-0643 Quadcore AMD laptop

So, my wife wanted an Acer Aspire laptop because of its size, primarily. The form factor was just right for her. But, it came with Windows 8, which she really hates (she loves Ubuntu Linux and Libre Office Writer). So, I downloaded Ubuntu Linux 13.04 Raring Ringtail and installed it on this Aspire V5 122P 0643 box. Complications?

1. windows 8 doesn't like to give up control of the BIOS. Going to system settings and boot from USB key (which I'd already put Ubuntu's install image on via another box).

2. Once installed, it wanted to reboot. I did so.

3. VAST multi-day hassles due to black screen after boot, then prompting for user login: instead of going to the graphical user login page familiar to all Ubuntu users. I messed around for a long time trying various things. Running startx failed with a message about no screens. Trying to reboot with some setting for recovery mode got me success once, but I couldn't reproduce it.

The command to show all hw installed revealed the video chipset is AMD Radeon 8280. This appears to be the only laptop, or in fact any device whatsoever, that uses this chipset, though Ubuntu seems to think it's on some desktops somewhere (in their compatibility pages).

4. FINALLY, solved the problem. On my other laptop, navigated to find the download for AMD's proprietary driver, and found the download destination.

Logged into Ubuntu using the user created during installation. Did 'sudo ls' and reentered my password so I had sudo privs without prompting. Then, downloaded the AMD driver from the link on this page: http://support.amd.com/us/gpudownload/linux/Pages/radeon_linux.aspx

the download link was http://www2.ati.com/drivers/linux/amd-driver-installer-catalyst-13-4-linux-x86.x86_64.zip, so I retyped it at the command prompt as:

$ wget http://www2.ati.com/drivers/linux/amd-driver-installer-catalyst-13-4-linux-x86.x86_64.zip

and it downloaded. I

then did: unzip amd* and then did: chmod 755 amd* then agreed to everything and installed it, rebooted per instructions, and voila! IT WORKED!

So, the answer is that the fglrx drivers (automatically installed as part of the amd drivers download, methinks) are the correct solution here.

The touch screen does work , but is of limited usefulness since the icons are so small in a normal linux desktop screen, as they should be. After all , windows 8 sucks at trying to unite the phone with the desktop environment, because they're disparate platforms and everyone except Microsoft seems to know this.

Enjoy!

How to Disable password prompt during ssh login with key failure

I wanted to login to a bunch of servers and test whether I could get in, or if I got error messages.

After messing around a while, I worked it into:

for n in `cat allhosts.txt`; do echo "host: ${n}"; ssh -oConnectTimeout=2 -oKbdInteractiveAuthentication=no -oPasswordAuthentication=no -oStrictHostKeyChecking=no -oChallengeResponseAuthentication=no myuser@${n} echo '----------'; done

Question on StackOverflow: Best Unittest for Python programs using OptionParser() ?

Just posted this on stackoverflow. http://stackoverflow.com/questions/17223530/python-how-do-i-usefully-unittest-a-method-invoking-optionparser