Guava's I/O Sources and Sinks on Android

I love Guava's I/O wrappers - ByteSource, ByteSink, CharSource, and CharSink. They encapsulate I/O stream (or reader/writer) access in a way that makes it easy for me to open & close the stream in the same location; when I pass a stream directly, the points of opening it and reliably closing it may be far apart. They also have helper methods to sweeten some common tasks, like reading an entire char source.

Android provides an existing mechanism for accessing private files while controlling their access modes: Context.openFileOutput and Context.openFileInput. As a Source/Sink primer, here's a cool way to adapt between the two:

The usage would look like this:

AndroidPrivateFileGuavaAdapter adapter =
  new AndroidPrivateFileGuavaAdapter("filename", Context.MODE_PRIVATE, context);

 // Read all text
String contents = adapter.asCharSource().read();

 // Write some bytes
byte[] bytes = ...; // obtains bytes from somewhere
adapter.asByteSink().write(bytes);

Android: backup and restore an app's data during development

During development, having an app with a realistic configuration and data set is really useful. Here's a trick that can help.

Connect a device with the app configured with lots of good data, turn USB debugging on, and then:
adb backup com.example.mypackage

Later, on another device (or the same device after a clean install):
adb restore backup.ab

W00t.

Python: change and restore the working directory

Occasionally a command should be run from a specific directory. Here's a Python helper for this:

• try/finally ensures the original working directory is restored.
• contextlib.contextmanager is a neat way to write things supporting the "with" statement ("Context Managers").

Hg vs. Git in the workplace

This post is based on the following mail I sent to the ALT.NET Israel list. I'm discussing a Windows development environment.

I'm much better at git than hg, and I use git a lot for real work, and it's great (no ifs or buts).
I also helped originate the move to git at work, because the transition plan was much better. I evaluated hg first and loved it, but the transition plan wouldn't have been half has smooth (a better hg-svn bridge may have been written since then).

Still, here are - in order - the reasons I thing hg would fit better at my workplace (and had we found a good transition plan we'd probably be using hg instead of git):

1. Friendlier commands and interface. With git I feel like I need to research how to do straightforward things. Git is extremely powerful and I can do anything with it, and I'm strange so I like the arcane, but git requires more of an investment from people to use it effectively when they have other things to do. Some things are arguably easier to understand in git (rebasing, for example, vs. hg's patch queues - although that's debatable) but the whole package leans heavily to hg IMO. This is the least-loved aspect of git among the teams.
2. In hg, commits remember the named branch they were committed to (see here). I would LOVE to have this in git, but no cigar - we're still playing with different schemes of approximating this.
   With many people working at times on several branches, and with merging done between branches, then given commit XYZ it's nontrivial in git to figure out which branch it belongs to.
   This is especially true in some scenarios we encountered - e.g. human error caused an older release branch to be fast-forwarded to the head of the latest devel branch - few traces are left in the repository itself of the previous state, and we had fun a few times fixing this (can take hours). Now we have a hook to prevent this.
   If we'd have cherry-picked between different branches instead of merged then this would be easy (and this is equivalent to svn's merging, only better), but we like git's merge and we want to use that.
3. hg has less quirky Windows support. Points to consider:
   • msysgit gets sporadic releases and is usually months behind the stable git version, the latest hg versions just work.
     • This hit us a few times (bug has been fixed but not available in msysgit).
     • Current example: Git 1.7.2 has improved crlf handling but msysgit isn't there.
   • Being able to "hg serve" your repository is incredibly cool, but "git daemon" doesn't work with msysgit
     • Think continuous integration that watches your local repository and runs builds with your current code, giving up-to-date feedback
     • collaborating with someone without pushing a publicly-visible branch (I've actually wanted to do that a few times)
     • developing and testing on more than 1 machine simultaneously. There are workarounds, but "git daemon" would have been perfect.
   • automating hg, including hooks, run in a Windows environment (vs. git where it runs in mingw's MSYS environment)

What doesn't bother me much for the long run is tooling. Right now hg seems to have better tools for our scenarios (even including much better Trac support), but I think git's extreme popularity will cause better and better tools to be written. Some people were very sad to let go of their VisualSvn / Ankh plugins, but they manage with Git Extensions (mostly) and occasionally TortoiseGit. I was surprised at how few people use git from the command-line, I can't live without it.

jQuery Validate: Required If Visible

I needed a generic "required if visible" rule for use with the jQuery Validation plugin. This is the result:

// jQuery.validate.requiredIfVisible.js
// Copyright (c) 2010 Ori Peleg, http://orip.org, distributed under the MIT license
(function($) {
$.validator.addMethod(
"requiredIfVisible",
function(value, element, params) {
function isVisible(e) {
// the element and all of its parents must be :visible
// inspiration: http://remysharp.com/2008/10/17/jquery-really-visible/
return e.is(":visible") && e.parents(":not(:visible)").length == 0;
}

if (isVisible($(element))) {
// call the "required" method
return $.validator.methods.required.call(this, $.trim(element.value), element);
}

return true;
},
$.validator.messages.required
);
})(jQuery);

Notes:

• Using required:"#myinput:visible" could have worked, but ":visible" doesn't check for hidden parents, and I didn't want to repeat the element selector.
• ":hidden" is false for elements with the "visibility:hidden" CSS rule, so I couldn't use ":not(:hidden)" either.
• Technique for checking element visibility from Remy Sharp's blog.

MVC's RenderPartial in a WebForms page

We have a mixed ASP.NET MVC/WebForms application, slowly transitioning to all-MVC. Obviously, sharing components between MVC and WebForms is important to us.

Using WebForms controls in MVC views is easy enough (minus the postbacks), but when we rendered partial views in a WebForms page using Keith Henry's code we occasionally got evil "Validation of viewstate MAC failed" errors. Microsoft doesn't think hybrid WebForms / MVC apps are relevant, but then Mauricio Scheffer came to the rescue. Instead of having the partial views subclass ViewUserControl, we now subclass Mauricio's ViewUserControlWithoutViewState - problem solved.

TeamCity 5.0 upgrade - "out of memory" error

I upgraded JetBrains TeamCity to version 5.0 on Windows (for Git support, yay!). The previous version upgrades went smoothly, but this time I got an exception when loading the server:

java.sql.SQLException: out of memory

After some googling, I thought I had to:

• Run ...\TeamCity\bin\tomcat6w.exe //ES//TeamCity
• In the "Java" tab change the value for "Maximum memory pool"
• Not set it too high

But no cigar. I realized that I probably had a corrupt HSQLDB database, and I ended up reinstalling and copying the config by hand (which was a little frustrating). I don't think there's a real reason why an embedded SQL database should be less dependable than one with a server (I know that from SQLite), so I'm guessing it's either TeamCity's or HSQLDB's fault. Even so, an "export configuration" button would be nice, and take the pain out of reinstalling TeamCity.

Encoding usernames and passwords in FTP URIs

I want to encode the following into a single URI:

• Username: 'jane+foo@example.com'
• password: 'my password' (with a space)
• ftp server: 'ftp.example.org'

The FTP URI that I expect:

ftp://jane%2bfoo%40example.com:my%20password@ftp.example.org

In .NET, HttpUtility.UrlEncode almost does what I need - it hex-encodes the special chars, but it also converts spaces to "+" which the FTP server interprets as a literal "+".

Final code that works with my tests using the FileZilla FTP Server:

string EncodeFtpUserInfoComponent(string s)
{
  // assume or assert s != null
  return HttpUtility.UrlEncode(s).Replace("+", "%20");
}

Console colors on Windows in .NET

Changing the console output color is easy on *nix terminals with escape sequences, but has a funky API on Windows (see the color support code from Testoob for a Python example with pywin32/ctypes).

It turns out that there's a very nice .NET API for it, though (there's a nice article by Sam Allen on the Dot Net Perls site). This would be awesome for Testoob's IronPython support.

Include literal file contents in ASP.NET

I was looking for something similar to Velocity's #include for ASP.NET. <% Response.WriteFile([filename]); %> works well, but a .NET control for this could be nice. Here is my attempt (MIT license):

using System.IO;
using System.Web;
using System.Web.UI;

namespace FooControls
{
  public class LiteralContentFromFile : Control
  {
    public string Path { get; set; }

    protected override void Render(HtmlTextWriter writer)
    {
      var server = HttpContext.Current.Server;
      var physicalPath = server.MapPath(Path);
      var fileContents = File.ReadAllText(physicalPath);
      writer.Write(fileContents);
    }
  }
}

Usage in an .aspx file (WebForms or ASP.NET MVC):

<%@ Register TagPrefix="foo"
  Namespace="FooControls" Assembly="FooControls" %>
...
<foo:LiteralContentFromFile runat="server"
  Path="~/path/to/file" />

Edit: subclassing System.Web.UI.Control instead of System.Web.UI.WebControls.WebControl which added unnecessary and unwanted behavior (for example, wrapped the output with <span>).

Issues API for Google Code Hosting

Google have recently released an issue-management API for projects hosted on Google Code.

That's good news for Testoob - I've been wanting to import its old Trac tickets for ages!

Testoob 1.15 released

I'm very happy to announce that after nearly 3 years Testoob sees another release.

Testoob is the advanced Python test runner and testing framework that spices up any existing unittest test suite.

Changes: Version 1.15 (Oct. 2009) adds better Python 2.6, IronPython, and Jython support, as well as test coverage improvements, better color support, and some new options and bugfixes. A full list of changes is available in the release announcement.

A big shout out to Ronnie van 't Westeinde - without him this version wouldn't have been possible.

Options to install:

• With easy_install: "easy_install -U testoob"
• One of the available packages

IDCC 2009 Registration Is Open

The response for the call for content for IDCC 2009 has been impressive, and now two tracks are open with great talks.

Hurry up and register!

"Enter" submits my ASP.NET form!

ASP.NET encourages web pages to be written as one big <form>, and every interaction with it to be a POST to the server ("Postback" in ASP.NET). This means pages often have multiple auto-generated <input type="submit"> elements.

On some browsers, pressing Enter causes the first submit button to be "clicked". In pages with many such buttons, that's usually not what you want. As a hack to make "Enter" do nothing on an already-built page, I added the following before any other submit button:

<div style="visibility:hidden;height:0;">
  <input type="submit" value="Dummy" onclick="return false;" />
</div>

Now this do-nothing, invisible submit button "catches" the Enter and prevents the next button from being "clicked". This requires JavaScript turned on, but ASP.NET's "Postbacks" require them anyway, so no big loss.

Implementation note: using 'display:none;' caused the browser to ignore it, so I had to resort to 'visibility:hidden;' and then 'height:0;' to impact the layout as little as I could. There could be cross-browser styling issues here, my specific case didn't need anything special.

Python: checking if an executable exists in the path

I wanted to do this on Windows. Based on this StackOverflow question and on Trey Stout's and Carl Meyer's responses I made the following snippet:

def exists_in_path(cmd):
  # can't search the path if a directory is specified
  assert not os.path.dirname(cmd)

  extensions = os.environ.get("PATHEXT", "").split(os.pathsep)
  for directory in os.environ.get("PATH", "").split(os.pathsep):
    base = os.path.join(directory, cmd)
    options = [base] + [(base + ext) for ext in extensions]
    for filename in options:
      if os.path.exists(filename):
        return True
  return False

JavaScript Namespaces

It's important to namespace your JavaScript code. I want one "master" namespace, and everything else under it (like YUI's namespacing approach). I fell into some potholes of browser compatibility on the way (with var foo; if (!foo) foo = {};), but here's what I ended up with:

// define master namespace
if (typeof foo === "undefined") var foo = {};

(function(){ // lexical scoping, allows "private" variables and functions

// define sub-namespace
if (!foo.bar) foo.bar = {};

// these don't leak outside
function helper() { /* ... */ } // "private" function
var counter = 0;                // "private" variable

// this function shows up globally under namespace foo.bar
foo.bar.func1 = function(x) { /* ... */ };

// not just functions
foo.bar.constant1 = 12;
foo.bar.shared_data = [];

})(); // end of lexical scoping

This way all the JS files can use the same namespace hierarchy, and can still be defined separately. No JS library necessary.

Fresh instances in Python with lazy evaluation

Testoob's --capture feature (written by Misha) replaces sys.stdout and sys.stderr for each test being run, and displays the output only if the test fails.

Leeor Aharon, of ff-activex-host fame, wanted to use it with Python's logging module, but the StreamHandler class stores a reference to the output stream on initialization - it already has a reference to sys.stdout, so replacing it won't affect it.

We originally thought of sublcassing StreamHandler and making retrieve the logger from a property, but we came up with this elegant code instead:

 
class LazyEvaluator(object):
    def __init__(self, factory):
        self.__factory = factory
    def __getattr__(self, name):
        return getattr(self.__factory(), name)

lazy_stdout = LazyEvaluator(lambda:sys.stdout)
handler = StreamHandler(strm=lazy_stdout)

The "lazy evaluator" is initialized with a factory callable, and every time it tries to access an attribute or method the object will be re-created by the factory. No changes necessary for StreamHandler, and ./alltests.py --capture works like a charm.

IE7 compatibility image wants to install IE8

I use Microsoft's IE Application Compatibility virtual machine images to test web sites with IE6 and IE7. Well, today my IE7 virtual machine wanted to automatically update to IE8. Great for the web, funny for an IE7 compatibility test image :)

Reddit and Digg icons on Blogger without external scripts

It took me a while to be able to add reddit and Digg icons that didn't load external scripts - Blogger's templating language had me stumped. I used reddit's, Digg's, and Blogger's documentation, but I would never have gotten the "onclick" escaping right without this post.

<!-- reddit button -->
<a expr:href='"http://www.reddit.com/submit?url=" + data:post.url' expr:onclick='"window.location = \"http://www.reddit.com/submit?url=\" + encodeURIComponent(\"" + data:post.url + "\"); return false"'> <img src="http://www.reddit.com/static/spreddit1.gif" title="submit to reddit" alt="submit to reddit" border="0" /> </a>
<!-- digg button -->
<a expr:href='"http://digg.com/submit?url=" + data:post.url' expr:onclick='"window.location = \"http://digg.com/submit?url=\" + encodeURIComponent(\"" + data:post.url + "\"); return false"'> <img style="background:white;" width="16" height="16" alt="Submit Story to Digg" title="Submit Story to Digg" src="http://digg.com/img/digg-guy-icon.gif"/></a>

P.S. - if you can't find the "data:post.body" element to add the buttons after, like I couldn't, see if you forgot to check the "Expand Widget Templates" checkbox under "Layout -> Edit HTML".

Prefetching JavaScript (or anything) with jQuery

While users are logging into a web site, I thought why not prefetch some JavaScript files they'll be needing on the next page?

I could load them with Ajax and this will be invisible to users (see the Even Faster Web Sites talk by Steven Souders, and his discussion of browser busy indicators).

The following seems to work, prefetching 2 files in parallel:

(function($) {
 $.ajax({ url:"/js/file1.js", cache:true, dataType:"text" });
 $.ajax({ url:"/js/file2.js", cache:true, dataType:"text" });
})(jQuery);

The 'text' dataType means jQuery won't try to evaluate the JavaScript it fetches. The 'cache' parameter defaults to 'true', but I prefer adding it.

I tested several browsers with a proxy, and the JavaScript files are cached - cool! I presume this technique would work with any resource, not just JS.