merging firefox bookmark files

I started to use google bookmarks. But syncing with it is rough, if not dangerous. The gmarks extension was not very helpful, though it could be used at points.

I wanted to merge my windows and linux bookmark files. Foxymarks is currently down, so I couldn't even try that. And I didn't like the fact that I HAD to sign up for their service. Just let me do it via the FTP option, and I'll handle it myself.

Finally I decided to make copies of each file, import them both into firefox, then using the duplicate bookmark detection extension in firefox, I was able to week out the dups quite easily. Still need a solution for syncing with google tho.

Rockbox database woes on the sansa; tagtool

Adding songs to the sansa with rockbox currently still has to be done via the original firmware (OF). But, everytime I would plug in the usb cable, (with the unit off to have it automatically start up and boot to OF), the OF would insist on "refreshing the database". This could take several minutes. The latest version of the rockbox bootloader fixed that. It now prevents an OF database refresh when the unit is powered up via usb. Great. Just had to run sansapatcher -d, then sansapatcher --install.

Now, the next issue was that sometimes the files loaded from linux would show up in the rockbox database, sometimes not. Turns out that many of the music files didn't have ID3 tags. So, in RB I changed it such that I could at least temporarily browse to the files to play them.

So, enter audio tag tool for linux. Excellent tool for updating/modifying ID3 tags en mass. Yet still the files didn't show in the database. I fiddled with various settings in RB to re-initialize, update the database to no avail. Finally I just deleted the database files in the root RB folder, restarted and all was well.

Rockbox on the sansa E280

Got the sansa last year for xmas. Hadn't used it too much. Wanted to play video, but the sandisk software for converting was pitiful at best and the highly uncommon video container and encoding made it near impossible to encode files in linux using something like mplayer.

Enter rockbox. Plays mpeg1 and mpeg2 videos, provides a highly customizable interface, and even plays gameboy games! Check out the manual for installation, etc.

My video conversion script ended up something like this:

#!/bin/sh
# convert $1 into an mpeg named $2 for use with the sansa

# options tried:
# 15, 24 fps
# 256kbps, 512kbps
# mpeg1, mpeg2

# 256k yields poor quality video
# 512k doesn't work with 24fps
# no noticeable diff for mpeg 1/2 - 1 is simpler, use it

mencoder "${1}" -of mpeg -oac lavc -lavcopts acodec=mp2:abitrate=128 -af resample=44100:0:0 -ovc lavc -lavcopts vcodec=mpeg1video:vbitrate=512 -vf scale,harddup -ofps 15 -zoom -xy 220 -o "${2}"

JanusVM, tor, privoxy, etc.

The need for anonymity and privacy lately has risen. The JanusVM vmware virtual machine is a simple entry into this area. This small vmx runs a whack of privacy things like tor, privoxy and squid. Just run the thing, and set your browser to http proxy localhost:8888. There are a myriad of other things that can be done with it, but this is usually sufficient.

vmware and a broadcom gigabit ethernet adapter

My Dell SC-420 has a Broadcom BCM-5751 adapter....any large-ish transfers from the host to a vmware guest is extremely slow, when it works at all!

These two posts point to issues with the vmware adapters and the tg3 broadcom driver:

http://www.vmware.com/community/thread.jspa?messageID=687712򧹠
http://www.vmware.com/community/thread.jspa?messageID=551754򆭊

They also point to the solution which is:

ethtool -s eth0 autoneg off speed 100
ethtool -K eth0 sg off rx off tx off tso off

That seemed to fix it....

sshd as a socks 5 proxy & foxyproxy

This isn't new stuff, been around for a long time and been written about many times, but too cool to pass up.

If you have an ssh account on a server somewhere that you trust, that connection can be used to tunnel all your network traffic securely out to the internet....main use case probably while at internet cafe's.

OpenSSH has built-in support as a socks 5 proxy server. So, using putty for example, in the "tunnels" configuration setting, create a new entry with source port being the port you'll connect to on your local machine, leave destination text field empty and select "dynamic" as the destination. Now, make an ssh connection to the server - netstat should show that your local computer is listening on the source port you specified.

Now you can configure your browser (or whatever other application supports proxies) to point to localhost and the port you specified. Check your IP address before and after at http://www.dslreports.com/whois.

Now, to make life super simple, install the firefox plugin foxyproxy to make proxy switching a simple single click!

general desktop and server news

well, it's been a while....

Dumped PCBSD as a desktop because after several upgrades I realized I was really only running FreeBSD with KDE on top. I wasn't using any PCBSD "benefits" and a somethings had gone screwy with the upgrades - particularly the Xorg updates that were broken along the way.

I also realized that I just didn't want to spend time configuring and playing with distros....this occurred to me because I attempted ArchLinux because it looked to be perfect for me....optimized for 686, and oriented much like BSD. However, it took some thought and maintenance to keep going.....so I abandoned that.

My requirements are few....firefox, python, and a shell. Add on top of that vmware server because I wanted to abandon the separate server and use just one machine. Has to be fast and light.....my desktop has a mere 8MB on board video RAM. Enter fluxbuntu.

I ran fluxbuntu for a while....it's quick and already has my fundamental requirements save for vmware, but that was pretty easy to install. However FB was based on dapper and the subversion client for dapper was too old for my repo. Two months into FB I decided to try the upgrade to Feisty and it failed in various ways. FB also had some annoying issues. There was supposed to be a Feisty release of it but that didn't happen. They are anticipating a Gusty release....maybe I'll try that.

For now I'm on Xubuntu, with vmware running my server. Having the server on vmware makes for simple backups of the entire vmware images onto dvd.

homemade DVDs in a dodge caravan

Last year we bought a dodge caravan with an in-house dvd entertainment system. Great! The kid loves the wiggles and if we can make a long trip manageable, all the better. However, it seems that all my homemade movies just wouldn't play....loss of A/V sync, choppy video, it'd take 5 minutes to read the disc, most of the time it'd just fail.

I'd done some reading that implied that I needed to use DVD+R media and set the booktype to ROM so that the player would think it was a pressed DVD. So, I tried that, but the player said there was no disc.

A bit more googling led me to a groups post that said something about menus.....none of the DVDs I create have menus....it's just easier without. So, I recreated a DVD with menus, and whaddya know? It worked. Odd that the mere presence of menus affect the ability of the player to decode video and keep sync.

UPDATE 2008-01-19: seems that the stupid player doesn't like DVD+R discs....stick with quality DVD-R!

freebsd printing with CUPS and a Samsung ML-2010

When I was running ubuntu last year I vaguely remember using my Samsung ML-2010 printer without too much fuss.....I think I found something in the excellent ubuntu forums that suggested using the ML-4500 driver....and it worked. Didn't always print nicely, but it pretty much worked out of the box.

So, yesterday I needed to print something....searching for FreeBSD+ML-2010 yielded little to no results, and nothing useful. In fact the top hit in google groups was a post by a guy I went to university with - but nobody responded to him. So, I embarked on my own journey. CUPS was already installed. I didn't really bother trying LPD cause I thought it'd be neat to print from any computer in the house via this box.

dmesg showed that I had /dev/ulpt0 - perfect. I started up CUPS, used my browser to view the web interface at port 631, and ran through the setup wizard to add my printer. But nothing samsung listed? So, I search the web for what to do next....turns out I need a PPD file for my printer.....samsung's website has one for linux....so I tried that.....completed the work in the web interface, print a test page.....and it complains. Missing some kind of filter? I guess I needed more than the PPD file.

So, keep searching.....splix a third party samsung driver.....but there's no freebsd port in /usr/ports :( As it turns out, there is one in CVS as well as several postings in the news group about creating the port files. After following the instructions in the PR report 111034
I had the port files.....but the Makefile had issues.....missing operator on line 34......there were spaces where there should have been tabs....copy paste error on my part. Forgot Makefiles were so damned picky about tabs.

So, I built the port, installed it....re-created the printer using the CUPS web interface. Print a test page and I get "Unsupported format 'application/postscript'!". Odd. Let me try from the command line......

/usr/local/bin/lp -dditdy-printer ubuntu and debian that claim this error was due to a missing file on those distributions with CUPS 1.2.3 - the file pstoraster.convs. Well, this file ain't on my system....."locate pstoraster" led me to /usr/ports/print/cups-pstoraster. Hmm....a seperate port.....wonder why something that seems so fundamental wouldn't be installed....? Well, I installed it, restarted CUPS tried the test page and no unsupported format complaint.

But the job sat in the queue. PR-99460 solved that issue: in printers.conf change deviceURI from usb:/dev/ulpt0 to file:/dev/ulpt0. CUPS won't be able to query the device but printing should work.....so I ran a test page after restarting cups and it appears to have completed (I did the test remotely, will have to wait until I'm home to see if it worked!).

Ugh....that was long and painful!


UPDATE:

I got home to find pages printed with errors listed on them. I can't remember the specific error, but suffice it to say that I'd used splix-1.0.1, and they'd recently released splix-1.0.1-1 specifically to workaround a firmware bug in the ML-2010. However, building this new splix might be an issue.....to start the tarball is half the size of the previous rev.

I did a diff on the two versions, and there were a few files changed as well as rastertospl is missing in the latest? I tried to build 1.0.1-1, but it complains about missing some tool ppdpo? Apparently that's part of the cupsddk, which I couldn't build either. OK, dig deeper.

A closer look at the diff shows that only one relevant file changed.....src/spl2.cpp had a two line change.....so I copied the file over to the 1.0.1 folder, repackaged the tarball, put it in /usr/ports/distfiles and tweaked the port's distinfo file to have matching MD5 and SHA256 signiatures and matching file size. Rebuilt and installed, printed a test page and it worked!

MOre FreeBSD upgrade woes - large FAT32 partitions

After the upgrade to FreeBSD 6.2 I could no longer mount my 250G FAT32 partition? mount_msdosfs was complaining that the partition was too large. Turns out there's a kernel configuration options MSDOSFS_LARGE that needs to replace the standard MSDOSFS option in the GENERIC kernel. I guess PC-BSD had it enabled in the GENERIC kernel. So, I rebuilt the kernel....kinda wanted to stick with the GENERIC kernel since I don't want to add anything more to the plate of things I have to remember to maintain, but in the end I guess it's for the better since I was able to disable a whack of stuff in the kernel and ideally it will run quicker/smoother.

CD/DVD burners lost after FreeBSD upgrade

So, after an upgrade to 6.2 k3b no longer detects my DVD/CD burners? After upgrading all my ports and packages (and breaking a few in the mean time), it turns out that atapicam wasn't loaded....must have been in the GENERIC kernel for PC-BSD. Whatever. Adding atapicam_load="YES" to my /boot/loader.conf should fix that.

Intel E7221/i915/i810 video on FreeBSD

My PC-BSD 1.3 installation was based upon FreeBSD 6.1. My Dell SC420 has Intel E7221 on-board video which is based upon the i915 which uses the i810 driver. However, dmesg told me that it was using the isa-vga driver. Things were sluggish....after a while, switching windows would take a few seconds. my Xorg log said that /dev/agpgart wasn't found? Sure enough, nothing. After much searching, it seemed my best be would be to upgrade my kernel and source as the latest i810 driver had made some significant improvements.

So, I upgraded to 6.2 - unfortunately this didn't help. I found several posts that said to hack vga_pci.c to always create an agp device. But vga_pci.c doesn't even seem to be in the source anymore?

JVC Everio and making a DVD

Last year when I was looking to buy a camcorder I discovered that there was basically three kinds on the market - miniDV, dvd, and hard disk. I didn't want to have to spend hours on end transcoding video from a miniDV cam as well as having to deal with adding a firewire card, etc. Nor did I want to have to waste a lot of money buying the dvd-ram media that a dvd camcorder would require. For me it's all about making the most of my time. So I bought a JVC Everio GZ-MG20 hard disk camcorder. The everio records video in mpeg2 format and audio in ac3. Perfect - already in dvd ready format. As well, the fact that the transfer mechanism is USB and the hard disk shows up as a mass drive, makes it even simpler.

Now - when the hard disk almost filled up I decided it was time to make some dvds. So, I installed the windows software that came with the unit - power producer I think. I fed it the whack of files....created some menus, and expected it to simply author a dvd structure for me. But instead it started transcoding the video - it was going to take several hours?!?! But the whole point of recording in mpeg2 is so that it wouldn't be necessary to transcode.

So, I searched high and low and tried several dvd authoring packages - but all insisted on transcoding. Until I found TMPGEng DVD Author from Pegasys software. It let me create a few menus and create a dvd structure and it only took as long as the vob structure took to write to disk.

I have two issues with this software: first, it only runs on windows.....now that my desktop is BSD.....the second issue is that I would have liked for the chapter names of each of the videos to be set to the filename automatically, or at least an option to specify.....I run a script over the video files that renames them to the date that they were created....for this purpose. TMPGEnc automatically names each chapter as chapter 1, chapter 2, etc. Manually renaming 100 files is tedious at best, so I have settled for letting it name them this way. I also keep raw archives of the videos on dvd and my server, so the filenames/dates will be preserved there.

There were a couple of other nice options it provided - there is the ability to create a DVD without menus, as well as having the first action (when inserted into a dvd player) to play all tracks sequentially (when using menus).

So, in the end, for windows, TMPGEnc DVD Author seems to be the best solution for authoring DVDs. The reason for this post is as a precursor to some posts I hope to make regarding my trials and tribulations of authoring DVDs on BSD/Linux.

splitting and joining avi files

I captured some old home videos from VHS on to my computer using my Adaptec Gamebridge 1400 (not recommended, will hopefully post about it later). I'd converted them to MPEG4 using mencoder and ffmpeg.

I had a file that ran too long....needed to chop off part. I loaded the file into vlc, figured out the time at which I needed to stop (1:35:32) and used "avisplit":

avisplit -i movie.avi -t 01:35:32.00

I also had a couple of files that needed to be one so I merged them using "avimerge":

avimerge -o outfile.avi -i infile1.avi infile2.avi

Note, the -i and files need to come last!

Both avisplit and avimerge appear to be part of the "transcode" package.

ssmtp on freebsd

I've been using FreeBSD off and on since version 3.x was considered leading edge. I never cared much for keeping close tabs on the logs and overall operation.....I'd update when I had the chance, have a look at the logs, see what's been happening. I found sendmail filling my logs to be quite annoying - I didn't want to be emailed cron output. So, I disabled sendmail in rc.conf.

However, I have little spare time and so having an email in my inbox each morning with a summary would now be useful. So I re-enabled sendmail, but as it turns out, as is with many ISPs nowadays, outgoing connections to port 25 are disallowed - so having my server send the email directly (to my gmail account) wasn't gonna happen. So, looks like the default sendmail configuration wasn't going to work. A bit of googling told me that sendmail configuration was gonna be rough. So....I looked for an alternative. Enter ssmtp.

ssmtp is a simple MTA sendmail replacement that I found while looking at the FreeBSD handbook. It provides simple configuration for sending outgoing mail only - which is all I want to do. This gentoo how-to was also quite helpful (as many gentoo postings are).

I made copy of the sample ssmtp.conf file in /usr/local/etc/ssmtp/ and added the following:

root=email_addr_to_send_root_emails_to
mailhub=isp_smtp_server
rewriteDomain=domain_emails_should_appear_from
hostname=computer_hostname
AuthUser=isp_email_login_id
AuthPass=isp_email_password


As I said, my ISP won't allow connections to port 25, so I have to use their smtp server, thus the need to provide authentication credentials.

Test with:

echo "testing mail service from my host" | mail root

And a few minutes later I see an email from my host.

rtorrent & dtach

My search for a new torrent client has come to a close with the winner being rtorrent.
It provides an ncurses interfaces and encryption! Perfect. No more having to deal with a resource hungry GUI. Unfortunately the port for rtorrent didn't include a default rc file or create on upon first run. So, I downloaded the rc file from the website, tweaked it to my liking and off we go.

During my search I stumbled upon people suggesting the use of "GNU screen" to be able to allow rtorrent to run in the background, yet be able to attach and detach from the application at will and from remote locations. Cool.

However, I also saw a lot of posts about GNU screen being bloated, slow and awkward to use - and most suggested dtach instead: a fork of GNU screen that has been trimmed down.

So after installing rtorrent and dtach and reading the man pages I gave it a go:

dtach -A /tmp/rtorrent.dtach rtorrent

The program starts....I add a torrent file (the backspace key)....and sure enough it runs like a charm. ctrl-\ and I've detached. dtach -a /tmp/rtorrent.dtach and I'm attached again.

dtach will definitely come in handy when remotely installing ports/packages or building something that will take longer than I can keep a remote session open. Sure a build can be done in the background, but portupgrade requires an attached terminal because it may ask questions of the user.

utorrent & *nix

So, I'd been using uTorrent for quite some time - excellent program! Extremely small footprint, efficient and well designed. Other than some of the sysinternals tools, I haven't seen a windoze program that is within the same class. My only beef with uTorrent is that it's not open source.

So, I recently switched away from windoze on my desktop....was on ubuntu for a while, but in the end I'm a FreeBSD kinda guy and thought I'd give PC-BSD a try. I really like it.

So, what to do for a torrent client? Azureus is a pig. I need encryption 'cause my ISP is throttling. Ktorrent? Didn't work to well....even with encryption enabled, my DL/UL rates were significantly worse than with uTorrent.

So, as it turns out, uTorrent runs on top of wine! Okay, so now my speeds are back up....but.....wine is taking up 40% of my CPU usage at all times! So much for a really efficient torrent client.

Next?