XDXA

FoundryVTT Module Test Automation

2023-02-19T20:57:00Z

I really loath manual testing. In a professional setting, nearly all of my projects are developed using TDD (test driven development). However, a bit to my embarassment, most of my personal projects haven't received this level of care. In this article, I'm going to overview how I bucked that trend by introducing testing to a friend's project.

Back in 2017, I started GM'ing the FFG Star Wars Tabletop RPG for my friends. Over the years, folks moved and eventually the game migrated online. In early 2021, a friend suggested I explore FoundryVTT. It quickly became our favorite tabletop RPG platform due to its extensibility.

We used the Star Wars FFG System, and shortly thereafter, my friend began development of an add-on module (FFG Star Wars Enhancements) with features not really suited for the core system.

Since then, FoundryVTT has undergone three major releases. While I've contributed a number of features to the module, the majority of the maintenance and testing during those releases has fallen on my friend's shoulders. After discussing options, I decided to use my background in CI/CD workflow development to setup automated testing.

Challenges

In many projects, a good mocking framework will give you sufficient coverage for testing against integration points with other libraries. In our case, FoundryVTT effectively acts as a framework for the module. This tightly couples the project's code with the implementation of the FoundryVTT API. The level of mocking that would be required to adequetly test a feature would prove extremely unreliable during a major version change. Many frameworks eventually create a testing scaffold that blends production code and fakes. This may be in FoundryVTT's future, but at time of writing it does not exist yet.

That leaves the next best option being integration and end-to-end tests. The difference between them is subtle. For the purposes of this article, integration tests attempt to minimize their reliance on UI interactions and end-to-end tests attempt to primarily test the UI.

Additionally, FoundryVTT (reasonably) restricts distribution of their software. That makes automating tests challenging, particularly in CI.

Quench

Quench is a FoundryVTT module that can be used to run tests from your module within FoundryVTT. This allows you to write Mocha tests and run them within FoundryVTT. We use Quench for our integration tests, where we try to avoid UI interactions.

Excerpt from a quench test:

it(`creates a new ${option} journal`, async () => {
    // Hook to capture when our dialog has actually rendered
    const rendered = $.Deferred();
    Hooks.once("renderApplication", (...args) => rendered.resolve(args));

    const dialog = await create_datapad_journal();

    const [application, $html] = await rendered.promise();

    // Sanity check the renderApplication hook returned our dialog
    expect(dialog).to.equal(application);

    ... 8< snip ...

    const datapad = game.journal.getName(option);
    expect(datapad).to.not.be.undefined;

    const page = datapad.pages?.values()?.next().value;
    expect(page).to.not.be.undefined;

    expect(page.text?.content).to.have.string(needle);

To see how we use Quench in action, check out our Quench tests.

Cypress

For full end-to-end tests, we've opted for Cypress. Cypress is a good fit for our project, because our tests really are purely frontend tests. Cypress's opinionated approach to testing encourages us to write tests in a way that avoids a lot of the patterns that make UI tests brittle.

We developed a number of Cypress Commands that handle installing systems, modules, and creating a test world.

Excerpt from a cypress test:

// Open the crawl dialog
cy.get('[data-control="ffg-star-wars-enhancements"]').click();
cy.get('[data-tool="opening-crawl"]').click();

// Create a folder for the Opening Crawl journals
cy.get(".window-content .yes").click();

// Create a crawl
cy.get("#ffg-star-wars-enhancements-opening-crawl-select .create").click();

Aside: If you end-up exploring Cypress, I strongly recommend reading their documentation. Specifically, Conditional Testing. Wrapping your head around the "Element existence" section will go a long way to avoiding common pitfalls.

Tests inside your tests?

An astute reader might have noticed that Quench requires a running instance of FoundryVTT to execute tests. To support running Quench tests in CI, we actually execute the Quench tests via Cypress.

Honestly, this feels a bit hack. But, writing integration tests in Quench feels more fluid, and this gives us a bit of the best of both worlds.

Docker Compose

Having the ability to quickly spin up and tear down different versions of FoundryVTT is extremely useful when testing. Locally, I'm using Docker Compose with felddy/foundryvtt-docker's docker container.

The container is really well designed and pretty flexible. To learn more about our Docker Compose configuration and usage, check out our Cypress README.md.

GitHub Actions

Tests that don't run as part of CI inevitably end up broken.

Aside: I actually worked backwards from this requirement to design all of the above, but explaining it in that order would have been very difficult.

Cypress makes testing in GitHub very each. They expose a github-action for running Cypress tests that handle launch and readiness checks for your server. Additionally, it automatically archives video recordings of your test runs.

To run FoundryVTT in a GitHub action, we reuse the learnings from Docker Compose. The GitHub Ubuntu runners come preloaded with Docker. It's easy enough to craft a docker run command that will launch FoundryVTT with our module's code installed.

From there, a few challenges remained:

Caching the installation to avoid abusive downloads of FoundryVTT
Securing FoundryVTT credentials while still supporting tests on user forks
Requiring approval only on PRs from forks
Race conditions!

Excerpt from our GitHub action configuration. See StarWarsFFG-Enhancements repository for all workflows.

- if: ${{ steps.container_cache.outputs.cache-hit != 'true' }}
  name: Launch FoundryVTT and run Cypress Tests
  uses: cypress-io/github-action@v5
  with:
      start: >-
          sudo docker run
          --name foundryvtt
          --env FOUNDRY_ADMIN_KEY=test-admin-key
          --env FOUNDRY_USERNAME=${{ secrets.FOUNDRY_USERNAME }}
          --env FOUNDRY_PASSWORD=${{ secrets.FOUNDRY_PASSWORD }}
          --env FOUNDRY_LICENSE_KEY=${{ secrets.FOUNDRY_LICENSE_KEY }}
          --publish 30001:30000/tcp
          --volume ${{ github.workspace }}/data:/data
          felddy/foundryvtt:release
      wait-on: "http://localhost:30001"
      wait-on-timeout: 120

More detail about the repository configuration can be found in the Cypress README.md.

1. Caching

To ensure we don't repetitively download the FoundryVTT application, we need to setup caching within the GitHub workflow. Fortunately, the felddy/foundryvtt docker image supports this natively. All we need to do is cache the data/container_cache directory. Unfortunately, the container entrypoint will still authenticate to the FoundryVTT API if a username/password is provided. Normally, this feature gives you a way to make sure you're running the latest release. In our case, we don't want that. To avoid it, we have two "steps": one if the cache hits and one if the cache misses. We omit the FOUNDRY_USERNAME and FOUNDRY_PASSWORD when we have a cache hit, skipping the check entirely.

To actually cache, we use the actions/cache@v3 action provided by GitHub. The only special thing we do here is an additional step that saves on failure to ensure we capture the installation regardless of whether the tests pass. The cache should hang around for at least seven days.

2. Security credentials

Getting the level of flexibility we wanted out of GitHub proved a little difficult. We wanted to know that tests would pass before merging a pull request. To support contributors forking our repository, that means checking out their code and running tests against it. However, these tests launch a docker container that requires secrets (FOUNDRY_USERNAME, FOUNDRY_PASSWORD, and FOUNDRY_LICENSE_KEY). To make those secrets accessible in workflows run with code from forks, we use the pull_request_target event.

This is potentially very risky. To limit our exposure, we use a GitHub "environment" on PRs originating from forks called requires-approval. The environment has a few contributors assigned to it that must manually approve every run.

3. Approval only on forks

Requiring approval only for PRs coming from forks was tricky to configure. The yaml SDL provided by GitHub actions is pretty flexible, but code reuse can get difficult. There isn't a good way to specify the workflow environment based on the origin. To work around that limitation, we create two jobs: one for upstream and one for forks. That splits the workflow across cypress.yaml where the jobs are defined and cypress-impl.yaml where the actual steps are defined.

This was a bit quirky to setup, but the sharp edges are pretty well documented in the configs.

4. Race conditions!

The free GitHub Actions runners are awesome! But they definitely run in a low performance environment. The heavy load will unfortunately/fortunately highlight every sloppy race condition in your tests. Adding fixed length timeouts generally is not reliable.

This makes tests trickier to write. You have to be very careful to be sure you've waited for the right thing before proceeding.

Additionally, some of our code, the system code, and FoundryVTT itself is not written in a way that makes it easy to know when you can proceed.

A few examples:

The close button on "Applications" have a 500ms timer before it attaches the onClick handler to prevent accidentally closing. Meaning, clicking it does nothing from fast automation.
FoundryVTT has Hooks for a lot of asynchronous events, but reasonably not everything. If you launch a dialog, you need to be thoughtful about how to know the dialog resolved. This could be done with a custom Hook, or by looking for a side-effect (like the creation of a Journal).

Summary

I hope that this proves useful to someone who hopes to setup test automation for a FoundryVTT module.

String Concatenation - Root of all Evil

2014-02-04T20:04:00Z

After college, I spent three years in software security acting as a pen-tester. It was my job to identify security vulnerabilities in the company's software via exploitation. During that time, string concatenation was the root cause of the vast majority of my findings. For that reason, it is my general philosophy that:

If you're concatenating strings, you're doing it wrong.

What?

To make my point easier to grok, let's start with a simple example:

>>> person.first_name + " " + person.last_name
'Bob Smith'

We've joined a person's first and last name with a space. The concatenated value can be thought of as a space delimited string. In of itself, this is not an issue. But, what happens when we need to reverse this transformation? Easy enough. Using the previous output as our input, we split on the space character.

>>> name = "Bob Smith".split(" ")
>>> print name
['Bob', 'Smith']

What happens if the first name contains a space? If an attacker controls that value, an injected space could break the application's intended behavior. It's conceivable to imagine that after deserialization, the value might be used in some security related decision.

>>> name = "Bob Smith RealLastName".split(" ")
>>> print name
['Bob', 'Smith', 'RealLastName']
>>> if "Smith" == name[1]:
...     print "Welcome Smith!"
...
Welcome Smith!

To address the vulnerability, we have a few options:

Validate the first name, rejecting requests with spaces
Sanitize the first name, removing spaces
Introduce an escape sequence and escape the names

Validation and sanitization are appealing choices, but if the business logic forbids it, our only option is escaping. A simple solution would be to replace spaces with '\ ' and backlashes with '\\'.

>>> person.first_name.replace(' ', '\ ') + " " +
    person.last_name.replace(' ', '\ ')
'Bob\\ Smith RealLastName'

You might be inclined to argue that this is a sufficient fix (nevermind that it's not to spec). However, I'm here to suggest that's not true. Manually escaping and concatenating strings is implicitly brittle. Forgetting to apply escaping algorithm in one place is enough to introduce a severe vulnerability.

The mistake here was casually performing serialization rather than utilizing some library, whose sole responsibility is serialization. This example is somewhat contrived, but the fundamental principal can be expanded to CSVs, XML, JSON, URIs, SQL, or any formatted data.

Examples

Let's break down a few exploits to demonstrate my point.

SQL

Here we have a simple SQL injection in PHP:

<? "SELECT * FROM table WHERE col='" . $input . "'";

If the input contains a single quote, "'", the SQL query's structure is disrupted. An attacker can use this to alter the query. (This example is benign. There are significantly more malicious uses for SQL injection, but that's not what I'm here to discuss.)

<?
$input = "' OR ''='";
$query = "SELECT * FROM table WHERE col='" . $input . "'";
echo $query;

SELECT * FROM table WHERE col='' OR ''=''

How do we fix it? There are three general approaches:

<?
1. "...WHERE col='" . str_replace("'", "", $x) . "'"
2. "...WHERE col='" . mysql_real_escape_string($x) . "'"
3. $mysqli->prepare("...WHERE col=?", $x)

The first approach manually sanitizes the input by blacklisting the single quote character. This approach is problematic, because it doesn't account for all possible special characters. If the surrounding quotes were instead double quotes, this method would fail. When I've seen this type of sanitization, it is typically done at the beginning of the request, far away from the code which actually utilizes the variable. That increases the probability of such a mistake.

The second approach utilizes a library call to escape the input. The last approach parameterizes the query, sending the query and its arguments separately to the database. Both the second and third approach utilize a library call, and both work for this query. If that's the case, what's wrong with using the version that utilizes string concatenation? What if the column was instead a number?

<? "...WHERE number=" . mysql_real_escape_string($x);

There are no characters to escape in this case! An injection like "0 OR 1=1" will still work. If the variable $x had been validated as an integer prior to use, there wouldn't have been an issue. However that's a brittle solution, because a simple mistake (or refactor) could still compromise the application's security.

What's really wrong here? The query is the formatted data. The library call is used to escape a value which is being serialized into the query. That's the responsibility of the library. By using string concatenation, we've effectively written part of a SQL serializer.

(Learn more about motivations to use parameterized queries.)

URI

The URI is a fascinating point to perform injection attacks upon. They are extremely easy to construct without a library, which means it's done frequently, and the results work most of the time. As more applications move towards service oriented architectures, URIs are increasingly used as a component of the communication. If unescaped input is placed into the URI, it might be possible to alter the application's intended behavior.

In the following example, we have a web application which transfers funds by dispatching to an HTTP service. The user controls the variable amount, but not user or target. The service domain s.xdxa.org is not internet accessible. We can assume the backend service will validate amount as a positive integer prior to use.

>>> print 'http://s.xdxa.org/pay?amount=' + amount +
                               '&from='   + user +
                               '&to='     + target
'http://s.xdxa.org/pay?amount=100&from=eve&to=bob'

To force Bob to pay Eve, we only need to alter the URI structure.

>>> amount = '100&from=bob&to=eve#'
>>> print ...
'.../pay?amount=100&from=bob&to=eve#&from=eve&to=bob'

The fragment (everything after the '#') is dropped before making the backend request. As a result, the service's log will look completely normal. To fix this, use a library to generate URIs. This is typically accomplished by using either a builder or providing a map of "parameter to argument"s.

Unfortunately, libraries for RESTful URIs, where arguments may be part of the path, are more scarce. In that circumstance, concatenation + escaping may be your only option. However, if you do that, segment that code into its own domain, i.e., create a library. The Java library Handy URI Templates is a good model for generating said URIs.

Additionally, be careful in other sections of the URI. Other components of the URI have different rules, i.e., scheme and authority. If you want to know more, The Tangled Web has an entire chapter dedicated to the URL.

HTML

XSS (Cross-Site Scripting) is a vulnerability where an attacker is able to inject JavaScript into a page, which is then executed in a victim's browser. It's a subset of the attacks against poorly serialized HTML. The attack is made possible by the fact that HTML mingles mark-up and code. In fact, HTML is composed of a number of different data formats within a single document:

HTML Tags
JavaScript
CSS
URIs

When represented in a object structure, we refer to the page as a DOM (document object model). However to get that page to the browser, it's serialized as a string and transmitted over HTTP. It is in that process that XSS is made possible. To make this concrete, let's take a look at an example. What happens if an attacker includes a script tag in the URL arg parameter? (Again, this is a simplistic example. There are significantly more malicious XSS attacks.)

<span><?php echo $_GET['arg']; ?></span>
becomes
<span>Hello! <script>alert(1);</script></span>

The user content altered the structure of the document. To fix this, we could escape the variable prior to output. However, that is simply string concatenation. The output is exactly the same.

<span><?php echo htmlentities($_GET['arg']); ?></span>
is the same as
<?  echo '<span>' . htmlentities($_GET['arg']) . '</span>';

This means, it falls prey to all the same risks as above. One missed escape, or escaping for the wrong context, and we have a vulnerability.

<span href="<?php echo htmlentities($_GET['arg']); ?>">...</span>
becomes
<span href="javascript:alert(1);">...</span>

But wait! XSS is really because a programmer failed to check for JavaScript within the user inlined content. That's true, in the implementation of HTML we know and love. If the issue wasn't truly with the failure to preserve the document structure, we could simply introduce a tag which would disable all scripts within the child element. That hasn't happened, because the issue is really about the injection. (As an aside, avoid using HTML for user supplied markup. It's a losing battle.)

So what do I recommend? Templating engines. (Keep in mind, templating engines may not automatically escape variables within your output. Escaping must be configured on a per-engine basis.)

<span>{{ input.arg }}</span>

That having been said, automatic escaping is typically not a panacea. Why? The escaping is normally context unaware, which means it's performed for HTML tags. If your templating engine allows bad markup, it's doubtful that it understands the context for which it's escaping. Where can this bite you?

Tag Attributes
URIs
CSS
JavaScript
(It is rarely safe to output content into a script tag. Avoid this whenever possible.)

Another even safer approach is to construct your page as a DOM. This will ensure tags, attributes, and possible URIs are properly escaped, but probably not CSS or Javascript. Almost no frameworks actually do this, because it's cumbersome and (probably) more cpu/memory costly.

Tangential from string concatenation, is a perfect templating engine enough? Unfortunately, browsers suck (from a security perspective). You could trace this back to the browser wars, where browsers which refused to render bad markup were labeled at fault. Regardless of the history, browsers have extremely "flexible" parsers. As a result, even if a document looks rightish, it might still contain an XSS. While I generally do not recommend sanitizing user input (it feels too much like a blacklist), input destined for HTML is an exception for the reasons outlined above. See OWASP's Sanitization Recommendations for candidate libraries.

You could easily write a book on injection attacks against HTML. For that reason, I'm going to stop here. Want to know more?

JSON

To avoid making this article much longer, I'm not going to break out a JSON example. However, I do want to mention it, because I have frequently seen engineers construct JSON from a string. Don't do this.

Embedded Variables

Note, there is no distinction between embedding a variable and string concatenation. Neither of the following two examples address the serialization problem:

<? "$first $last" === $first . ' ' . $last;

Or with sprintf like substitution:

"%s %s" % (person.first_name, person.last_name)

Logging

Before we conclude, I want to make a special note with regards to logging. In my code, casual string concatenation most frequently occurs in logging statements. Logs are typically loosely formatted, newline delimited files. Loosely structured output is mostly fine when intended for human consumption. However, this casual serialization format is a nightmare for log parsers.

For this reason, one must be careful when logging output which will eventually be consumed by some downstream process. While not strictly the responsibly of the logger, HTML log viewers have been susceptible to XSS attacks in the past.

See CWE-117 and CAPEC-106 for more information.

Clarification

This concept is not new or innovative. The weakness I'm describing is actually CWE-707: Improper Enforcement of Message or Data Structure, and it is not exclusive to string concatenation or even strings. I wag my finger at string concatenation, because it makes these mistakes easy. And consequentially, common.

Hyperboles

The title is mostly meant as tongue and cheek, but I'm quite serious in the severity of this topic. In my opinion there are only two places string concatenation should occur:

Serialization/Deserialization Libraries
Unformatted Data

Many security issues simply do not have an easy solution, e.g. side channel attacks. However, if engineers consistently use libraries for serialization, the majority of injection based attacks will simply go away.

Is it possible to write secure code with concatenation? Yes. However, this code will almost always be brittle. You might understand the security considerations, but the next engineer probably will not. Writing secure code isn't just preventing a certain type of attack. It's following best practice throughout the entire code base to set the standard which avoids careless mistakes.

Vertical Rhythm

2013-11-17T17:54:00Z

I've always found interface usability fascinating. Subtle differences in coloring, spacing, and layout can drastically impact the functionality of your application. I think it's important for programmers to understand some of the basic principals of usability.

A few years back, I read about a design concept referred to as Vertical Rhythm. The way I like to think about it is to imagine a web page as a sheet of lined paper. All text is written upon the lines, which is referred to as the baseline. A heading could pass through multiple lines, but it shouldn't impact the following text's ability to align with the baseline.

Why go through the effort of creating a repetitive baseline? People are extremely good at pattern recognition. If we utilize this skill set, our interfaces will feel natural to use. When skimming text, a consistent baseline makes it easier to read, because the text lands in the location expected by our brains.

I set out today with the intent to write a script to assist verification of a page's baseline, but it seems someone has already accomplished this. Baseliner provides a bookmarklet. When executed on a page, it will overlay a series of horizontal lines. These lines can be adjusted to match your page's line-height and layout. At the time of writing, this page has a grid size of 21 with an offset of 15.

Like all things CSS, numerous styling issues can arise. When styling your web application, keep the following in mind:

Top margins will collapse into previous tag's bottom margin.
Images may not be a multiple of the line-height.
Headings whose line-height is not a multiple of the baseline line-height will cause issues if wrapped.

When I initially discovered vertical rhythm, there wasn't much written on the web covering the subject. That has changed. If this interests you, I suggest check out the following:

How important is this really? One of my favorite browser extensions, Readability, which reformats poorly laid out blogs, doesn't maintain vertical rhythm. Nor does Wikipedia. Ironically, neither does the Baseliner page or the two links above. In fact, I couldn't find a single site that followed this advice.

Does that mean it's wrong? The lack of a consistent baseline on the web could be due to the difficulties in establishing rhythm with CSS. Or, it could be that there are more optimal spacing practices. In the end, I don't really know whether it's right or wrong. Regardless, this principle isn't dogma. It's a tool, just like the color wheel. For someone who is not a designer, I find it useful in establishing an application's foundation. Next time you're working on a layout, give it a try and see what you think.

Vagrant

2013-04-28T11:21:00Z

I love the notion of a clean, repeatable install. In my side projects, I've historically accomplished this through strict documentation practices and directory organization. However, this process inevitably breaks down for me when a machine becomes multi-purpose.

Either locally or in the cloud, I'll end up installing packages, hacking together configuration, or otherwise making the environment murky. In fact, the server that currently hosts this blog is a perfect example. This instance hosts this site, riiga.net, and has been a sandbox for toy projects I've worked on over the past two years.

While I'm reasonably confident I can redeploy this machine within a couple of hours, it would be far more flexible if I could do this in minutes. Or even better, selectively re-deploy projects to alternate hosts.

For this reason, Vagrant has become my new favorite toy. Vagrant provides simple VM construction and provisioning. This enables easy provisioning of development environments, testing large system upgrades, and migrating projects to alternative cloud providers.

To begin exploring the world of automated provisioning, I decided to create a VM that would serve the Mozilla Sync server. I've hosted the Vagrant Mozilla Sync Configuration on my GitHub. (Disclaimer: this was a warm-up project. All dependencies are automatically pulled from other source controls. Meaning, this will likely break in the future.)

Isn't this overkill? It certainly seemed so... that is, until the first VM corrupted. When that happened, I simply re-provisioned a new instance. About an hour later, I was back up and running with minimal engineering effort.

What next? I've been itching to try Linux Mint. And what better way than to automatically provision my standard desktop deployment!

Acrylamid

2013-04-13T19:09:00Z

For a long while I've wanted to host this blog rather than utilize a service like Google's blogger. My biggest hesitation was blogging software tends to be a common means of getting pwned. I could write straight HTML, but then the content is mingled with the markup - it feels dirty. FrontPage you say? Well, that's just not cool enough.

Enter Acrylamid, a simple, but powerful means of generating a static website. Out of the box, it supports:

Markdown
Jinja2 templating
Code highlighting with Pygments

To convert the contents of the old blog, I downloaded it using Google Takeout. Through a combination of xpath, html2text, and some python hackery, I converted the previous entries to markdown.

For styling, I'm utilized the responsive template from Initializr. While I've used both Bootstrap and their HTML5 reset in the past, this is the first time I've gone responsive. It's surprisingly straight forward and seems to work well on my android devices.

But wait! Where are the comments? Sorry folks - no comments. I debated about using a service like Disqus, but I just dislike the idea of including a third-party service (and consequentially tracking) on my site. If you have feedback, see the about page. There are numerous means of communicating with me there.

CakePHP's error handling

2011-02-17T09:04:00Z

Whenever possible, I like to trigger the appropriate HTTP status code for an error. CakePHP has a mechanism to throw 404 errors by simply calling:

<?php
$this->cakeError('error404');

This is handy when a request to a view action passes an id that doesn't actuallyexist. However, what about 403 or 500 errors? If you're like me, you probably stumbled upon AppError in the CakePHP book.

I defined a simple error403 method, and used it where appropriate throughout my application. But when I rolled out to production, every 403 became a 404! Upon a bit of digging around in the code, I found the root of the problem in the ErrorHandler constructor. In version 1.2.9, you can find this in cake/libs/error.php:

<?php
if ($method !== 'error') {
  if (Configure::read() == 0) {
    $method = 'error404';
    if (isset($code) && $code == 500) {
      $method = 'error500';
    }
  }
}

(Side note: Although in production error500 technically can be called, $code doesn't appear to be declared in the constructors scope. Also, the method error500 isn't actually defined, which adds to my confusion.)

In production, all methods translate to either 404 or 500 errors! Now let me note, this does not appear to be mentioned anywhere in the book... So, how do we resolve this? Well, I found someone having a similar problem.

Teknoid sets debug to 1 when the error handler constructor is called. This causes the error handler to believe it's in debug mode and run whatever error method is passed its way.

But, there's a very important distinction. Teknoid overrides his _outputMessage method to send an email to himself, not to display errors to users. If you were to simply set debug to 1 in the constructor, all CakePHP errors would be displayed to the user. This is why only error404 is allowed in vanilla CakePHP.

To resolve this, I created a white-list of allowed error handlers. When these error handlers are requested, I enable debug mode.

<?php
class AppError extends ErrorHandler {

  var $allowedMethods = array(
    'error403',
    'error500',
  );

  function __construct($method, $messages) {
    if (in_array($method, $this->allowedMethods)) {
      Configure::write('debug', 1);
    }

    parent::__construct($method, $messages);
  }

  function error403() {
    // I do some logging here to audit
    // who made the forbidden request

    header("HTTP/1.1 403 Forbidden");
    $this->controller->set(array(
      'name'  => __('403 Forbidden', true),
      'title' => __('403 Forbidden', true),
    ));

    $this->_outputMessage('error403');
  }
  ... error500 and so on ...
}

This is less than ideal, because turning on debug may have other implications throughout your application. We could reimplement (or copy and paste) the constructor with allowedMethods in mind, but that feels even more hack to me.

Asus P4C800 Rev 2

2009-11-18T09:49:00Z

This board has been my latest personal hell. Back in the year 2005, my Asus P4C800 Deluxe fried. After taking their time, Asus sent me a replacement board. I simply assumed that it was a Deluxe and reinstalled it. Unfortunately, it was actually a P4C800 Rev 2. There are three models of the P4C800: P4C800, P4C800 deluxe, and P4C800-e deluxe. From what I can tell online, the Rev 2 is actually the e deluxe, but it looks like a hacked together mix between the P4C800 and the e deluxe.

This is where I made a mistake. One night when trying to get suspend to work in Linux, I decided to flash this board. I flashed it without checking what bios version was already on there. Stupid. Worse, since I knew it wasn't the deluxe (because the updater rejected the bios), I assumed it was the e deluxe. As would be expected, after the flash, the system wont boot. Flash forward a few days, and I've managed to force a flash downgrading the system to a P4C800.

Now looking back, I honestly can't remember what the board was originally and which bios I first forced onto the board. Never flash bios when tired and in a rush. I still can't believe I did that. Anyways, the system suffered from random shutoffs. I assumed this was due to the flash. (Honestly I can't be sure one way or the other now.) Over time these flickers grew increasingly annoying. So, I tried another flash. That time I managed to brick the board.

Yesterday I received two replacement bios chips. (One is a backup for hot swapping.) The latest version of the P4C800-e deluxe bios works, but it wont reboot. The latest version of the P4C800 bios works and will reboot. Both suffer from the intermittent shutoffs, and both have bios errors referring to the IRQ routing table. At this stage, I'm pretty sure the problem either isn't a bios issue or at least isn't a bios issue that I can resolve.

I installed this board into a new case with new hardware and the shutoff problems ensued. This has brought me to the conclusion that the problem has to be with the board. Removing all hardware except the bare bones still results in the shutoff. Another irritation is the problem wont normally occur when the system is idle. Also odd is the problems happen most in XP, less in Ubuntu, and the least in Windows 7. I don't know what that means exactly, but it suggests that whatever is triggering the problem may be a deprecated technique that modern OS's don't use. Though it may be that the board is just getting old and needs to be tossed.

However, the two things that keep bring me back to suspecting the bios are: the coinciding timing of the first shut offs / bios flash and the fact that the board looks like a P4C800 with a Rev 2 sticker slapped on it instead of a P4C800-e deluxe board. There might be some bios out there that will stabilize this system that I have yet to find. I should probably contact Asus.

Although I'm sure this isn't the conclusion to this story, it should be noted that suspend still doesn't work. :(

Edit - 28/10/2010: This is the conclusion, because the pieces are now sitting in a box. My brother upgraded and let me keep his old hardware, which is newer than the P4C800. I'm now looking for a good way to get rid of this old hardware.

GBA Development on a Mac

2009-10-30T08:19:00Z

In my media devices architecture class, we are developing on the GBA. Unfortunately, most of the students are developing on windows machines, which leaves me and a few others the odd men out. On macs, we're using devkitarm as our compiler with a provided makefile. Unfortunately, the Makefile had a few flags misconfigured.

The stock makefile worked perfectly until we started dealing with sound. The problem was two fold:

Setting the interrupt handler would cause the GBA to reset
Sound files larger than thirty seconds would overflow the ewram

There were two macros in the make file that were used for compiling and linking respectively:

MODEL = -mthumb -mthumb-interwork
SPECS = -specs=gba_mb.specs

The model affects how the assembly is compiled, and the specs defines the way the linker will link the c objects together. I figured out that mthumb wasn't needed by looking at the windows makefile. Removing it resolved the rebooting issue.

I knew that the overflow had to do with the linker, because that's where it failed. I was at the point where I was actually trying to modify the linker script to manually force the audio memory to go to the right region when I realized there was a second gba .specs file. Switching to that file resolved the issue.

The resulting makefile macros are as followed:

MODEL = -mthumb-interwork
SPECS = -specs=gba.specs

I am not a compiler expert. So, I can't tell you the full implications of changing these flags, but it did work for me. I'm still working on what I believe to be a compiler bug and plan to submit a bug report soon. If it turns out to be interesting, I'll make another post here.

Microsoft Natural Ergonomic

2009-10-26T17:09:00Z

I decided I wanted to enable the zoom key on my "Microsoft Natural Ergonomic Keyboard 4000" under Linux. Ideally, this would be used for scrolling up and down a page instead of triggering Compiz's zoom feature.

Unfortunately, the kernel fix is triaged on ubuntu's bugtracker. This of course meant recompiling the kernel manually (or semi-manually since Ubuntu makes this task relatively easy). You'll notice, if you care to check, that there is a patch in the bugtracker to remap the zoom and spellcheck keys to something that xev (and therefore X11) can read.

If you don't know how to recompile your kernel, this fix probably isn't for you. However, should you choose to proceed anyways (or if you're just not familiar with recompiling your kernel under Ubuntu), you can find instructions for recompiling Ubuntu's kernel here.

Once you've downloaded and uncompressed the source, edit "/usr/src/linux- source-*/include/linux/input.h". Search for ZOOMIN, ZOOMOUT, and SPELLCHECK and change these constants to match the following:

#define KEY_ZOOMIN 246 /* AC Zoom In */
#define KEY_ZOOMOUT 247 /* AC Zoom Out */
...
#define KEY_SPELLCHECK 235 /* AL Spell Check */

Keep in mind that if some other input device already uses these key codes, then you'll have a conflict. Otherwise, once you've recompiled your kernel, you should be able to access these buttons in Gnome now.

Additional Information

What does this do and why do we do it? From my understanding, the reason this is a problem is X11 can't read key codes above a 255. Although this is a bug with X11, we can modify the kernel to remap these keys to a value lower. This enables X11 to read them and thus Gnome to use them.

I dislike patch files for problems that are not going to be fixed, because they often break on the next kernel release. This is why I chose to manually edit the file instead of offer a patch file

If you recompile your kernel, you'll likely have to recompile any other third party drivers you have installed. For instance, I had to recompile my Nvidia driver after rebooting. I also had to purge the nvidia-common package.

Mapping to Page Up / Down

Next, I wanted to map 246 and 247 to page up and page down respectively. To do this, I edited my ~/.Xmodmap file to include:

keycode 254 = Prior
keycode 255 = Next

You can activate this by running "xmodmap ~/.Xmodmap" or adding the command to startup applications. However, the next time you log in, I think you'll get an option to enable this automatically without having to add it to startup apps. (I haven't setup xmodmap in a while, because I already use it to swap caplocks and escape... yes, I'm a vim user.)

Edit: I should note that I have since reverted to a stock kernel. For some reason the recompiled kernel would kill my mouse periodically.

Mac OS X Snow Leopard Port Wine

2009-10-21T04:56:00Z

Currently, wine is not happy with my 64bit upgrade to Snow Leopard. Just for the record, I find no fault in either port or wine for this, because I was an early adopter. I upgraded port with a simple reinstall. (I know there are more elegant solutions than this, but I was being lazy.) After upgrading port, installing wine resulted in the following error:

Error: wine 1.0.1 is not compatible with Mac OS X 10.6 or later. Until wine 1.2 is released, please install wine-devel instead.

This is pretty straightforward. Obviously, the next step was "port install wine-devel". However, I was promptly stopped with another error in the following format:

Error: You cannot install wine-devel for the architecture(s) i386

Error: because /opt/local/lib/libjpeg.dylib only contains the architecture(s) x86_64.

Error: Try reinstalling the port that provides /opt/local/lib/libjpeg.dylib with the +universal variant.

Error: Target org.macports.extract returned: incompatible architectures in dependencies

I'm not too familiar with port or the complexities of 64bit architectures working with 32bit binaries, but I do understand the principle behind OS X's universal (fat) binaries. After googling around a bit I discovered that the way to compile in the universal format is with the following command:

sudo port upgrade --enforce-variants jpeg +universal

The part that hung me up was figuring out which package was associated with that file. After digging around in the man page I found the following solution:

port provides /opt/local/lib/libjpeg.dylib

That's it! Just keep installing the dependencies and retrying "sudo port install wine-devel". After about eight dependencies are recompiled, you'll be finished. Good luck.