Dec 3, 2012

OWASP Press

I have long desired a bookshelf full of OWASP books; and indeed other people have as well. As we have an OWASP Lulu store. The problem with this model however is that the publishing is remains oriented around the 'dead-tree publishing' workflow. Where books are written to a deadline and 'released' to the editors, and finally sent to the printers for publishing and distribution. And by the time the documents make it to publication they are already out of date.

OWASP has long provided some of the very best documentation on the web for Application Security. Among the awesome1 documentation resources are projects like OWASP Cheatsheets, The OWASP Development Guide, OWASP Testing Guide, OWASP AppSensor and OWASP SAMM.

These community documentation projects are living, and so they present a special challenge to the 'dead-tree publishing' workflow. Additionally, I have the books I love most on my Kindle not a physical bookshelf, who wants to lug a dead-tree when you can haul around a library?

The answer was obviously just-in-time publishing, after all isn't this what a wiki is? Where anybody from the OWASP community could update a text and the update be instantly available? While wiki technology really excels at on-line living documentation and knowledge management; wiki's have failed to succeed at general purpose publishing platforms.

I have spent the last month exploring ways to instantiate a just-in-time open publishing workflow to produce digital and dead-tree editions of the OWASP documentation projects.

Every avenue I turned down, I would inevitably run into numerous issues attempting to create a workflow. While, I was able to get some ideas to work successfully, I was unsatisfied, as the complexity required would have created a participation barrier to entry.

Fortunately, I was not alone in my thinking and I discovered last week that the people at Leanpub had solved exactly these issues that needed to be overcome to make this a reality.

Leanpub

Leanpub is a lean workflow applied to documentation. Publishing is indeed lean; as in Toyota lean; in fact it is just-in-time like the Qin Dynasty's terra cotta warriors! In fact they have written a very interesting manifesto about the future of publishing that I found to be very insightful and well researched. I highly recommend reading it if you are interested in such things.

While Leanpub is full of coolness for authors, I want to talk about how it works. Leanpub works via dropbox and plain-text markdown files.

After joining the site, you create a book by clicking a 'Create a lean pub book now' link; which takes you to a page that asks for the book title and a URL; and you submit the form with a 'create my book' button. Leanpub then publishes a manuscript folder to dropbox and shares it with you, and sets up a storefront for the book at the url you specified.

The manuscript is written in markdown2, However, since markdown is a simple markup format it doesn't have any file inclusion features, so everything is expected to be in a single file. This is a problem because it is very difficult for many authors to make changes to a single file, and also because it is simply nicer to separate out ideas that belong together into chapters.

This is one of the issues I ran into when attempting to build this for OWASP myself. I tried using makefiles and GPP to overcome this limitation of 'everything in a single file.'

Leanpub's answer is a simple textile called book.txt that simply lists the files in the order they are to be include as such:

about-the-book.md
acknowledgments.md
introduction.md
chapter-01.md
chapter-02.md
chapter-03.md
appendix-01.md
appendix-02.md

And with that the problem is solved, simply and elegantly. And unlike my attempts, it solves it without having to acquire and compile software to participate in authoring. Keeping the barrier to participation very low.

Dropbox's killer feature is sharing. However, while sharing is awesome; it is best used for distribution of information that doesn't change often. Additionally, it while dropbox keeps a complete history of files; dropbox doesn't provide the granularity of history that a good version control system does. This makes sense because sharing is about distribution of mostly 'static' content.

However, what is needed for massive collaboration, is good distributed version control software which is built with changes in mind. Particularly where you can have an infinite number of authors. You need something that has merging of conflicts built into its DNA. The coup d'état of distributed version control systems is Linus Torvald's git project. And the best place to host a public git repository is arguably github.

Github

Git is an amazing tool for community collaboration and change management so the final step is putting the shared dropbox under git version control and pushing the repository to github. Incidentally, OWASP has a github repositoryy and members of the community should be encouraged to use it.

In this example the dropbox and the github fork are on the same machine, However, they could easily be different machines; and in practice they will be as each author will have the github fork and only the project leader will have the dropbox.

To start an OWASP Publication, with the browser of your choice:

  1. Create a book at lean pub. (in this example AppSensor)
  2. Create a new repo at github. (In this example AppSensor-Handbook)

In a terminal:

  1. cd ~/dropbox/AppSensor
  2. git init
  3. git remote add origin https://github.com/OWASP/AppSensor-Handbook.git
  4. git push -u origin master

Everything is now ready for mass collaboration and realtime publication! In order for Authors to participate in the publication process:

  1. Download your favorite github.com client for mac or windows (or use the command line if your are so inclined).
  2. Clone the github.com repository.
  3. Fork your own branch.
  4. Edit the manuscript.
  5. push the changes.

Once the authors are done making changes or through the use of a git branching model. The project lead who has the dropbox computer will need to goto the dropbox folder and 'git -pull' the changes from authors at github. This will automatically syncs the leanpub website which shares the dropbox folder. The project owner can then publish the updates by navigating to the leanpub website and clicking the 'create a preview of my book' on actions tab.

While I appreciate that this is somewhat manual, github has service hook's that could be used to automate this workflow completely. I am currently exploring the best way to do this. If the dropbox is hosted remotely as well then the 'git -pull' can be automated. This need to be followed by a publish request with leanpub; and while it has some authorisation tokens and oner 'anti-automation' cookies; it would be trivial for most of the OWASP community to issue the post request to leanpub in an automated way.

OWASP Press

I have been calling this concept the "OWASP Press" project. And I hope you will join me in making it a reality, lets write and publish OWASP books in real time, lets do documentation similarly to a software project; lean and agile. :-) And, I hope to have you all keeping me very busy (and out of trouble on the leaders list…) updating the OWASP github repository with a bunch of OWASP documentation goodness!

As a final note, where OWASP books are concerned. It is my opinion that the funds generated from the book sales, should go directly to the associated books project. This way the project has a measure of its success (grand total sales) and it can hire out cover art, or have leanpub will produce a Lulu publishable PDF.

Additionally, I feel that OWASP publications should always be freely available. And finally, let the world know where to get your OWASP projects book as PDF, mobi, and epub; and invite the world to participate in authoring it!


  1. Indeed there is so much good documentation that the very act of listing example projects, is a failure to highlight some other equally awesome project. The fact is that there are many, many great documentation projects at OWASP and I encourage you to explore it for yourself. My apologies to the many incredible projects that did not come immediately as examples to mind when I wrote this. 

  2. Markdown is a very cool plain-text format that can be compiled by tools like pandoc into numerous outputs; including latex, mediawiki, doc, and html to name but a very few. Using pandoc the very same markdown text can easily be turned into mediawiki content for the OWASP wiki. 

#owasp   
Return button