Category Archives: development

FEE.org Project Retrospective Part 1: Six Secrets to Better Productivity for Independent Developers

As a software developer, I have balanced working as a developer/tech lead/architect, etc for various large corporations with working as a an independent IT consultant, building and running software for a number of small organizations.  Having the freedom to experiment in my personal work as well as exposure to professional enterprise practices has led to useful synergies, but balancing multiple projects has been quite difficult at time.

What follows are three posts with a retrospective of my latest project – porting fee.org to the Umbraco CMS, covering:

  1. Advice on personal time management for independent developers
  2. How I chose Umbraco as a CMS for this project
  3. Best-practices and tools for managing independent software projects.
  4. Content migration, custom feature implementation, launch planning, performance optimization, Amazon Web Services integration, and team engagement

I will cover the technical details of my latest project in the next post, but first, here is how I manage my time:

Speaking personally, the most difficult aspects of my work are 1 prioritizing tasks, 2 staying focused on them, and 3 work-life balance. At times in my career, there were times when I would spend only an hour or so per day working and the rest reading Wikipedia and debating in online forums. (By now, Wikipedia probably makes up over 80% of my general knowledge.)  For most technical people who don’t work for top-tier software teams or are much more productive than their team average, it’s quite feasible to spend only a minority of their time doing their assigned work. But that’s not a very smart to spend one’s career.  Anyway, here are my probably overly simple solutions for a more productive life as a developer:

1: Make setting priorities the first thing you do, and write down tasks for every activity

Setting priorities is the most complex activity that human brains engage in, and it is also what suffers the most when our brain is fatigued. We’re just not very good at setting and remembering lists of things to do. (see “Your Brain at Work” by David Rock).  So whether it’s for a six month project or a single day, I always start by writing down a list of tasks to be done and prioritizing them.

It used to be common for me to get to the office with a specific task on my mind, yet leave six hour later having worked on something entirely different. The solution to this is to making writing a prioritized set of tasks for the day the first thing I do every time I come to the office. I put task lists on paper or in Evernote, so I can refer to them later.

2: Use Project Tracking Software
On a professional level, I use project tracking software (JIRA) to write down all my tasks.  I have used JIRA to manage projects with 25 developers and multiple teams as well as projects where I was the sole developer – task tracking is essential for any project.  Using project tracking tools has many benefits for facilitating interaction with project stakeholders (that I will cover that in part #3), but it is quite essential for me to have a prioritized list of tasks for me to work on.

3: Use Test-Driven-Development

Following my task driven approach, I write the list of tasks to be done directly on my code. When I started as a developer, I would write down a list of comments in a file, then implement each function below, so that the task would become a comment describing what a single function did.  Now, I keep my classes small, and keep the big picture in a unit test class. This (1) helps me stay focused on what needs to be done (2) allows me to test each step in isolation (3) helps to detect regressions in the future (4) all the usually extreme programming benefits you can read at Wikipedia.

4: Write less code, spend more time on GitHub

It’s well established that developers on average write 10 production lines of code per day. So, if I find myself doing a lot of typing, I will step away from my computer and go for a walk. My goal as a programmer is not to write the most code, but to maximize the business value I deliver with my time.

I do this by spending my development time searching GitHub, NuGet, apt-get, etc for the best components, trying to fit them together, and building supporting tools (like unit tests) — and not debugging code or building components from scratch. If I really have a unique and worthwhile module to create, it is typically either a candidate for an open source GitHub project, or is some sort of highly proprietary business process/algorithm.

5: Schedule regular breaks

I have a theory that my obsessive Wikipedia and Reddit surfing was actually my brain’s cry for help for not allowing myself to relax and review my work. Unfortunately, by swiping from a mentally taxing production to a mentally taxing consumption activity, I was still not allowing my subconscious to relax and integrate.

I’ve since learned that it is important for me to walk away from my desk at least once per hour. When I’m deep down in work mode, I lose the ability to access, prioritize, or to identify when I need to try another approach. I will take loops around the office, or around the block several times per day to allow my mind to relax and review the approach I am taking to solving the problem at hand. It is very rare for me to be stuck on a difficult problem and have a flash of insight at my desk. I need to walk away to process problems on a high and/or subconscious level.

6: Use tracking analytics set goals, to improve time management – and do billing

I use RescueTime to track ALL my computer time, both work and entertainment. I use WakaTime to track the time I spend in IDE’s.  Git repositories record all the code and documents that I create.

Once a month or more, I spent several hours doing billing, which consists of opening up RescueTime, WakaTime, SourceTree App (git log), and Tempo (time tracking product integrated with JIRA) side by side. It takes me several hours to process 30 days, but I get a very detailed account of how I spent my productive and personal time. Once I understand how I’m spending my present time, I can decide what changes I want to make in the future.  RescueTime allows me to set personal goals for how much or little time I want to spend on certain activities.

Below you can see my allocation for July and August, and a single day.  I use all three tools to account for my hours.  Although only a portion of my work involves writing code, RescueTime helps me understand how much time I allocate to each project.

Monitoring OS X battery usage

Speaking of power usage, there are two nice apps to help monitor battery usage in OS X. CoconutBattery is mostly useful for checking the health of the battery.

 

MiniBatteryLogger is a more advanced tool that monitors power consumption and levels over time. You can use it to see how particular configurations are consuming power.

Another, slightly related app is fseventer. fseventer shows filesystem access in real time and provides nice insight into what apps are using the hard drive.


How often do you read code?

It is very possible and probably common to go through a major team development effort without spending very much time reading other people’s code. It’s possible to spend an entire career like that. If over 90% of the code you read is your own, how will you learn to be a better programmer?  No matter how many thousands of hours you spend programming, if you are doing only what you already know, you will not improve your skills. You must challenge yourself and sometimes fail to improve.

How can you read other people’s code?

  • Read your team members code
  • Read (& refactor) your old code – you probably forgot what you did already
  • Read books on software & algorithms
  • Participate in open source projects
  • Read blogs on software
  • Write a programming blog

Why fix bugs with unit tests?

I know this is nothing new, but I will share my perspective anyway:

How I fix bugs with unit tests:

  1. Write a unit test that reproduces the bug so the test fails
  2. Fix the bug so the unit test passes
  3. Send a link with both to the bug submitter

Why I like it:

  • Saves time debugging – just run the unit test instead of recreating the bug scenario every time
  • Creates a simple yes/no criteria to verify whether the bug is fixed
  • Regression can be detected quickly in the future
  • Better communication: bug is documented for other developers looking at this code
  • If you don’t have time to fix the bug immediately, write a failing test.  Then you or someone else can fix the bug later
  • An organic way to increase code coverage for existing code

Project highlight: Mises.org Multimedia CMS

In February, I did a high-level review of the Mises.org CMS admin UI.  Today I want to highlight both the public and private parts a part of it: the multimedia CMS.

The Mises.org Media Management System is part of the Mises.org open source project.  It was started around 2004, and evolved slowly from there, with the latest release around February.  It is used to manage over 6400 audio/visual files.  Total document size is 275GB.  Monthly bandwidth exceeds 10 terabytes.   (You can find  the source code at Bitbucket)

The front-end features:

  • Dynamic, AJAX-based interface build with JQuery UI
  • Advanced search and browsing based navigation
  • Detects the best media player for the current browser: HTML5, Flash, or Silverlight
  • Extensive RSS support.  Integrates with iTunes and iTunes University with iTunes-specific feed extensions
  • Optimized for scalable deployment, CDN’s, SEO, external embedding, and social network sharing

The media asset management includes:

  • Support for internal and externally hosted content: MP4, MP3, WMA, streaming video, & YouTube
  • Integrated drag and drop uploader: management interface allows uploading multi-gigabyte media files straight from the browser us
  • Meta data parsing: reads full meta information such as embedded thumbnails, duration, keywords, description, etc.  Able to process all popular audio/video formats as well as PDF.
  • Catalog builder:  organizes files in a human-friendly format: http://library.mises.org/media/
  • Define arbitrary hierarchical categories
  • Versioning and multi-volume support.

Architecture:

  • Backend: ASP.Net 4.0, MVC3 (frontend) & Web Forms (backend), Entity Framework, and SQL Server
  • Frontend: JQuery UI, HTML5,

How to get it for your site:

  • It’s part of the Mises.org CMS
  • See http://wiki.mises.org/wiki/MisesWiki:Development for details

Screenshots:

Continue reading “Project highlight: Mises.org Multimedia CMS” »

Tips for adding optional SSL support

I am excited by the growing popularity of the HTTPS Everywhere plugin. I became aware of it recently when users wrote me to complain that some part of Mises.org did not work over SSL. It turns out that our store
software redirects visitors to the official hostname, so going to https://mises.org/store would redirect to http://mises.org/store, which HTTPS Everywhere would redirect back to https://mises.org/store, and so on. I decided to fix this and also to fix the “this page includes other resources which are not secure” warning.

How I implemented automatic SSL switching:

  • SSL proxy: We use CloudFlare as an SSL proxy. Even though we already had SSL configured on our Windows server, CloudFlare makes it easy to share SSL certificates between Windows and Linux servers, which is otherwise quite tricky. But it’s also great if your web server does not have SSL support.
  • Relative (scheme-less) Urls: I removed the scheme from all our resources, so they look like <img src=”//images.mises.org/Theme/images/bf_lvmi.png” alt=”” />  We are transitioning to scheme-less URLs for everything.  It allows users to choose HTTPS even if your site is served over HTTP by default.
  • Request.Url.Scheme: I used to append the URL scheme used by the request when I was not sure if the client (rss reader, email, templates etc) would support scheme-less URL’s. But this has the disadvantage of not working with cached pages. So far, it seems that scheme-less URL’s are safe for all popular web clients.

I don’t think there was ever much sense in having the http://www. prefix in URI’s, so getting it down to just // is very nice.

How to find non-SSL resources on a page:

One frustrating debugging step for me used to be finding the http resources within a page. I would have to search for http resources in the source code. If the page references scripts that load other resources, this could get complicated. Fortunately, Google Chrome offers a simple solution – the network inspector can export information to a searchable JSON file:

  1. Open a new private browsing session in chrome (to disable your extensions, which may inject non-SSL links within the page)
  2. Alt-click and “Inspect element”
  3. Go to the Network tab
  4. Open the URL to inspect
  5. Right click inside the Network tab and “Copy all as HAR”
  6. Paste the JSON into notepad, and search for http:// resources

One last note: If you use third-party widgets or services which do not fully support SSL, please take the time to contact the vendor and request that they add full support for SSL.

Success factors for quality software development teams

Some notes from a short presentation I gave on building quality software development teams:

  • Effective teams start with good people. A business is not a charity and cannot wait for people to grow into the role required of them. Intelligence and inherent motivation cannot be taught. These and many other aspects must be screened for.
  • But hiring good people is not enough.  Getting the most of a development team requires certain environmental factors. I have organized them into three areas:  internal/motivational, structural/organizational, and interpersonal/communication. Sometimes they will conflict with practical considerations, but we ought to take them consideration whenever possible.

Motivation
  • Instill a sense of professional pride
    • Developers should take part in the technical design
    • Provide interesting, challenging work
    • Developers should own the technical solution
    • Technical architects should be part of the development team
  • Respect the developer time
    • Provide a quiet environment dedicated to development work
    • Buy the best tools for the jobs – powerful computers, large monitors, etc.
  • Reward learning and exploration
    • Take (reasonable) risks with new technologies
    • Schedule time for self-education and information sharing
  • Create a sense of project ownership
    • Delegation “ownership” of functional parts to individual people
    • Assign developers responsibility for follow-up maintenance (don’t just hand it off to a maintenance team)
Structure
  • Hold people accountable for their work:
    • Use a work unit tracking system (TFS, etc)
    • Make task status publicly visible
    • Hold daily stand ups (Scrum)
    • Consider a public scrum board
    • Provide clear project requirements
    • Lean development approach
      • No time-wasting tasks (useless documentation)
      • Isolate developers from unrelated tasks (no business interruptions)
  • Hold high expectations
    • Monitor quality of work with code reviews
    • Regular training & information sharing sessions
    • Consequences for bad work
  • Set well defined deadlines
    • Don’t set arbitrary deadlines, but all work should have a deadline
    • Developers should have input on deadlines for their work units
Team Communication
  • Instill a sense of collective ownership of the project
    • Hold architectural education sessions
  • High-bandwidth communications
    • Entire team should work in physical proximity
  • Make quality visible
    • Use continuous integration and automated testing to provide immediate feedback of quality
    • Developers should fix their own bugs
    • Track quality over time
  • Explicit mentor roles
  • Mentor roles should be explicitly defined
  • Mentoring time should be scheduled into the project