How I got started hacking on Firefox

One of the Gecko Layout & Rendering team’s main responsibilities is the continuing development of CSS in Firefox. I recently modified the CSS style system to allow nested rule parsing. This bug fix taught me a lot about the CSS parser and how styles cascade through the rest of the Layout engine. It took me a little while to set up a dev environment, understand the bug, write tests, get the code written, reviewed, and checked in. I now have a much better understanding of what it takes to move code through the Mozilla source trunk.

I suspect that there are other programmers out there currently lurking around the Mozilla community, intimidated by the scale of the source tree, and wondering where to start hacking on Firefox. I highly recommend starting with the Gecko Overview started by L.David Baron, Mozilla Principal Software Engineer, to help beginners understand the browser engine. Thanks, David!

Firefox Invalidation

As promised, I’m going to call out Firefox Layout & Rendering code changes that my team has been working on. This bug fix from Bas Schouten (as reviewed by Robert O’Callahan) was one of those light-bulb code changes that have really advanced my understanding of the Gecko Layout Engine’s graphics code:

https://bugzilla.mozilla.org/attachment.cgi?id=574190&action=diff

This check-in implements “Paint Flashing”, a diagnostic tool that shows when the browser is invalidating (or repainting) a screen region that has changed. It’s very useful for diagnosing redundant (painting too often) or greedy (painting too large an area) invalidation. Both kinds of bugs are notoriously hard to find.

Paint Flashing will paint a random color (at 20% opacity) to indicate the “damaged” area of the screen that the browser is repainting. You can enable Paint Flashing by setting this preference to true:

nglayout.debug.paint_flashing

If you’re running a DEBUG build, you can get to it in the Layout Debugger. The code is deceptively simple but shows the logical bottleneck where invalidation occurs in Firefox. We’re already using this tool as we evaluate larger invalidation changes that aim to dramatically speed up rendering in Firefox. You can also use this tool as you develop your web apps to see if performance issues are the result of invalidation problems. You can tune your app much better if you know how much screen area is getting painted from your javascript or CSS code.

A Different Perspective

I thought I’d try something new on this blog and invite a guest writer to post an article about the recent Mozilla Festival in London. I wanted to feature a non-industry perspective on this blog, which has seemed to veer too far into the technical side of things lately.

The article, MozFest for a Non-Engineer, is written by Jo Villegas. She currently lives in the SF Bay Area and spends her time writing and working at a cafe and a spice shop. She will be in London next year to pursue a degree in literature from Kingston University.

MozFest for a Non-Engineer

The Mozilla Festival was quite fun. Even though I am from a library and foodie background, and not an engineer, I found plenty of interesting workshops to attend. Over the course of two days I was able to learn about Hacker Journalism, video editing with Popcorn, How to get more  out of blogging with WordPress, How to better connect parents to take advantage of educational resources in their community,  how to promote your talents and credentials through a fantastic social media idea based on “Scout’s Honor” and I was able to meet one of my long time celebrity crushes….Tim Hunkin, the adorable brainiac host of ” The Secret Life of Machines ”

Hacker Journalism discussion centered around what journalists can do to remain true to the maker idea without compromising integrity. Much lively discussion ensued. It seemed to be an understood dilemma that if a journalist were to insist on being labeled a hacker, they would be in danger of trying to be the newest shock-jock, prone to bias and easily dismissed as a… well, “a hack.” Most journalists, who have actually gone to school and learned the ropes through years of experience would probably cringe at the label. However, there are many concerned citizens who may not have a voice due to censorship and government control of their local media. These citizens want change, and social media has enabled them to appeal to a larger audience in a matter of minutes.

Journalism, which started as a public service but quickly became profit driven, will continue to play an important part in the intellectual growth of our society. There will always be a mix of established traditional values regarding investigative and data driven journalism.

The idea that a revolution can be started through social media such as Twitter is not impossible, but it would serve as the tipping point. When a society has been made aware of injustices through traditional forms of journalism it slowly builds momentum for change. The people wait for that one event to serve as a spark to get everyone out of their comfort zone and into the street. The challenge for journalists who use social media in reporting is transparency, where the subject must be based on factual info, which changes almost instantly as witnessed during the London riots, Occupy Marches, and catastrophic events.

In conclusion, the group agreed that “Hacker” or more appropriately described “Citizen Driven” journalism will provide tons of relevant info, but it will take traditional investigative techniques to gleam the truth and mainain integrity for long standing journalism such as books and the encyclopedic records that become the real history of an event.

Further discussion on this topic is available by contacting the moderator, Nikki Usher.

The introduction to Popcorn video editing software was very interesting. If you are watching a video online and need to know more information about what is being said or shown at a precise moment, you can pause the video and read a pop-up comment that was inserted by the editor. This is a fantastic feature especially for educational videos because the viewer is able to click and get the footnote immediately rather than segue off to Wikipedia.

The software offers a multitude of tools to make a video a completely personal experience. Should I ever have a need to make an educational video, I would definitely use Popcorn because it offers many features and it is not too difficult to learn how to implement them into a video.

As a newbie in the blogging world, I felt it would be helpful to attend and introduction to WordPress workshop. The instructor, Mark Simpkins, started out with about fifteen participants, although it whittled down to myself and a very nice professor from Sri Lanka, as people realized that it was an introductory workshop. I now have a new enthusiasm for blogging and trust that WordPress will stand the test of time as a valuable tool for getting my two cents out into the world. Even if you are well established WordPress blogger, it doesn’t hurt to learn about how other bloggers are using WordPress, so do consider attending one of the WordCamps when it passes through your location.

There are many choices for parents if they want to broaden the extracurricluar activites of their children out side of school supported sports programs, but it can be difficult to know what programs are available in the community and how reputable the organization is that sponsors the programs.

HIVE London and beyond is a online meeting place for parents and community organizations to meet and share information. Parents can go through the website and find a multitude of programs available for their children ranging from art and music at a local museum or conservatory, to enrichment programs that enable children to learn about the diversity of the city the live in. It is difficult for traditional education systems to fully prepare children for global citizenship. HIVE is trying to establish itself as the “go-to” website for not only parents, but any organization that has an enrichment program that would benefit children who are yearning for new and interesting experiences.

A large part of the  HIVE vision is to promote cultural awareness in children, a noble effort on their part and in theory, should not be too difficult. Much discussion was had on what obstacles could appear. A website can only provide information, it will need community support in order to succeed. Participation by children is a given, parents need to maintain interest and enthusiasm or kids will feel their efforts are inconsequential.

Diversity and tolerance programs need to be more than just feel good measures. More work will need to be done so that socio-economic barriers are softened to encourage participation. When there is less marginalization among neighborhoods, it will allow children to socialize more freely. I believe all parents want safe and interesting activities for their children and HIVE is a great way for neighborhood organizations to promote themselves and share their unique qualities.

Mozilla Open Badges enable a person to show off their skills and experiences along with the rest of their shared online personal history. This sounds like a wonderful idea for people who have learned many skills outside of their traditional education. Based on the Scouts/Campfire reward system, anyone can earn a “badge” to show that they have mastered a skill or complete a number of related badges to show they have mastered a skill set. Badges go on a virtual backpack and can be linked to a Facebook or other social media so that friends can see your accomplishments.

With a recent grant by the MacArthur foundation, along with development and promotion by Mozilla, Open Badges is able to reach out to numerous schools and educational organizations that would give badges for completed coursework. There is also the opportunity for an entrepreneur to promote their business as a place to earn a badge. For example, I could earn a badge in intermediate yoga, cupcake decorating, or pet CPR by taking the class or workshop. Badges could be a boost to many people who have grown bored with their online social profile. Once there is a system in place that integrates the earning of badges with a social media forum, there is a lot of potential for positive peer to peer relationships. It is hoped that this concept truly comes to fruition and that it can be made so that it is manageable and achievable for anyone interested.

Tim Hunkin was the keynote speaker at the conclusion of Mozfest, and I personally believe that I was the most enthusiastic fan present. When I was a teenager, I would watch his tv show The Secret Life of Machines with my brothers. His voice is one of the easiest for me to recognize, right along with Anthony Daniels’ C3PO and Leonard Nimoy’s Spock. Tim gave a very entertaining slideshow about his arcade full of humorous themed contraptions, followed by an encouraging speech about what being a “maker”  is all about. I felt that he really drove home the idea that one doesn’t have to have a million dollars or even what would be considered a million dollar idea in order to create and make something fun or interesting…just find a way to get the project going and don’t stop until it is complete, then don’t be afraid to show it to the world.

If you have an idea for a fun mechanical invention, but are stuck on the details, find Tim for a bit of advice.

Flash: I’m not dead yet!

My last post generated a fair bit of discussion around the true fate of Flash within Adobe. The original announcement from Adobe and the associated staff reductions had caused a lot of speculation within the Flash community. I should clarify some of my statements, especially the parts about the Flash Pro authoring tool and Flash Player.

All software follows a similar life cycle from 1.0 to a final version when it effectively becomes frozen in time. I didn’t mean to imply that Adobe is pulling Flash Pro from store shelves like salmonella-tainted spinach. A familiar pattern is to move software engineering to a low-cost geography like India or China as growth slows. This is not a bad thing, and actually lets the software have a long tail of maintenance and incremental feature work at a cost that allows it. The consumer can still count on that software being available for some time with support (usually from the same low-cost geography.) A handful of engineers will remain in the original geography to consult the new engineers as the hand-off occurs. The Flash Pro team in the US will have a handful of engineers left behind to manage the hand-off so, technically, the entire team is not getting eliminated at this time. Over time, the idea is to have your higher-cost engineers migrate over to your higher-growth projects. This is what occurred many years ago at Macromedia when Director moved offshore and the team moved over to working on Flash.

Last Tuesday’s announcements did not follow this pattern. Instead of moving the mobile Flash Player to a low-cost geography, it was discontinued outright. Instead of moving many affected Flash Platform engineers to the next high-growth agenda, they’re getting sent home. These events are what prompted me to write my last post as my friends at Adobe were blindsided by the whole affair.

I have a great deal of respect for the Adobe product managers who are trying to contain the damage and reverse the Elop Effect from the original announcement. It does seem like they were not made aware of the Tuesday announcement in advance. I sincerely hope they will be successful at affecting change within Adobe to keep Flash around for a good while longer.

Did Adobe finally kill Macromedia Flash?

Adobe announced that it is ceasing development on the Flash Player for Mobile Devices. As you may know, I was on the engineering team that brought Flash Player 10.1 to Mobile Devices back in June, 2010. This announcement is disappointing for many reasons, but not surprising given the realities of mobile web development.

While I was at Adobe, we spent a great deal of time and energy to get the Flash Player to run fast enough on the HTC Nexus One running the Android OS. At the same time, Apple was making it very clear that Flash would never be allowed to run under Mobile Safari, especially if our goal was to have it work on Android first. The writing was already on the wall even before we shipped: no matter what we did, or how well we did it, we were not going to have our ubiquitous platform extend to the Mobile world.

Of course, that’s not the whole story. What I’m truly disappointed about is that Adobe is exiting the Flash Platform business–they laid off 750 people yesterday, including the entire US-based Flash authoring tool team. This means that offering a free Flash Player runtime subsidized by selling tools is no longer a business Adobe is interested in. I always worked to ensure that SWF as a data type could have the same ubiquity as JPEG. That is, you could trust that the SWF you authored in 1998 would be rendered by a publicly available runtime in 2028. For this to happen, you need to have an organization that can shepherd the technology and commit to doing so for all time.

Unfortunately, Adobe is not that organization.

Or is it? I hope that they release the runtime code to the public domain so that others can extend and maintain it. I’m well aware that there are 3rd-party licenses in the code, but that can be easily redacted. If there is any hope left for Flash to survive, it will have to be an open-source implementation that can fulfill the promise of a ubiquitous rich-media data type for years to come. Adobe needs to acknowledge that the public needs SWF to survive, and that opening up the platform is now the only way to ensure that outcome.

Mozilla Festival, London UK

I will be in London from November 4-6 to take part in the Mozilla Festival. The festival will be at the Ravensbourne College in Greenwich (in the beginning of time—GMT.)

The organizers are actively looking for more developers and designers to add rocket fuel for the Festival’s roster of design challenges and hack sprints. More specifically, they are looking for more:

  • Javascript developers, HTML5 video and audio enthusiasts, front-end developers, news app developers, and other news hackers, to help build everything from data-driven journalism kits to mobile news apps to amazing browser-based video games that run right in the browser.
  • User experience and interface designers, graphic designers, game developers, and 3D modeling people, to help create everything from data visualizations to translation workflow to whole new ways of imagining news and information.

Know anyone who fits the bill? Please ask them to join us in London by signing up here: https://donate.mozilla.org/festival-register

If you have people in mind who would be perfect but could have a tough time paying the event fee, please send me a note. We’ll find a way to get them in.

Thoughts on Scrum

I’ve worked on many software projects over many years. A few of them went on to become billion dollar businesses and some of them never shipped. Some of them shipped after a great deal of pain and suffering. I learned a few things about software development processes that work and don’t.

Scrum doesn’t work.

I’m sure the Certified Scrum Masters will be quick to point out that I must have done Scrum incorrectly or somehow did not follow through with the methodology, and I’ll say that’s exactly why it doesn’t work: A whole industry of people who walk around to talk your ear off about Scrum and why it’s the best thing since the silicon wafer.

Here’s what I know to work:

  1. Generate a strong product vision.
  2. Hire the most talented software engineers.
  3. Make sure they understand #1 and let them build it.

Scrum aims to provide a safety net in case you fail to do any or all those 3 steps. Here’s the problem: you will not be successful unless you succeed at all 3 steps.

Scrum states that we can change the goals and vision based on lessons learned during the iteration. In practice, this leads to leaders with no vested interest in achieving audacious goals. “Oh, it didn’t work. Let’s try something else.” How are we going to get Teleportation with that attitude?

Scrum assumes that you can calculate your team’s velocity based on historical data and therefore predict your ability to deliver. This assumes that any slug that grows arms can write so many lines of code in an “ideal day.” It marginalizes talented software developers who don’t need to be forced to determine how many “story points” there are in their work. How many story points do you think transporting a kilogram of carbon across an ethernet cable would be worth? How many story points to do it wireless?

Scrum lets you review the work completed during an iteration and choose to continue pursuing missed goals or decide to pull something else off the backlog. Sounds great, right? In practice, this just hides incompetence and frustrates strong engineers. The lame engineers will come with lame backlog items and burn down to zero every time. The strong engineers will aim for the moon, and sometimes fail, and get shafted because they did not achieve the stated goals for the iteration. They’ll eventually leave.

I’d like to hear your thoughts on this, as I’ve seen Scrum spread throughout Silicon Valley like a locust plague. If you’ve adopted Scrum in your team, I’d like to hear about it:

  1. Was the product any good?
  2. Was it worse/better before you started Scrum?
  3. Did you ship on time?
  4. Was your team happy doing it?

Graphics Engines in Firefox

One of the key goals for the Firefox Layout & Rendering team is to improve performance for web applications. Web apps nowadays push the graphics rendering stack in ways that rival console games. To that end, our existing rasterizer was hitting its limits in terms of the sheer volume of data we now push through it on a a regular basis. Switching out a shipping application’s raster engine is a big deal that requires a lot of scaffolding. When I was working on the Flash Player, I saw how much work goes into integrating a new graphics engine into a shipping product. Sometimes, you have no choice but to tack it on top of (or in Flash’s case, behind) the existing rasterizer.

For Firefox, the scaffolding involves the addition of the new rasterizer initially for the  <canvas> API’s. This was ideal because it was fairly localized, and backwards-breakage risk was manageable. In the next phase, all CSS layout styling will use the new engine. The lessons learned from <canvas> make this a much less risky phase but it’s still fraught with peril. The benefits will be worth it, especially when we get page-draw and responsiveness metrics cranked up.

I’ll be posting up more details on Layout & Rendering internals as I dive deeper into it. It’s quite exciting to work on the guts of the Web with passionate engineers.

Shipping already? I just got here!

Firefox 7.0 shipped today. While I can’t take any credit for any of it, my team deserves all the praise. They’ve been hard at work on reducing memory all over the product. I’ve had a few people complain to me about “pork” in Firefox and we’re making some big changes to cut the fat. It’s interesting how getting a new job just means new complaints from the same cadre of “stakeholders.” Let me know via the blog comments if Firefox is working better (or not) with all these optimizations.