Since 2014, I’ve worked in digital preservation, building a large-scale distributed digital repository where a number of US universities can preserve valuable digital materials. In most of these universities, the library is responsible for digital preservation, and most libraries are understaffed. Many also have high turnover among their digital preservation staff, which means institutional knowledge is regularly walking out the door. Small staff and high turnover lead to a big problem: libraries often don’t have the staff or the skill to manage all of their digital preservation responsibilities.

When I joined APTrust in 2014, my mission was to create a digital preservation repository to be shared by a number of universities throughout the US. Universities would upload their most valuable digital materials for preservation, and our service would store them in S3 and Glacier, on the east coast and on the west coast. On top of that, we provided a number of additional services, including regular fixity checks, deletion workflows requiring admin approval, PREMIS event logging (PREMIS is a standard from the US Library of Congress), automated restoration spot tests, and more.

I started programming in the late 1990’s, when I was working at Amazon.com. Back then, the members of Amazon’s customer service department had to write their own code to get whatever information they needed to do their jobs. When customers called or emailed to ask where their book was, we ran home-grown Perl scripts called customerstat and orderstat to pull information from an Oracle database and display it in a Unix terminal.

For the past nine months, I’ve been working at APTrust, building an online digital repository for universities to back up important data. The main point of the system is to enable universities to recover data in the event of a local or regional disaster. The two big projects I worked on prior to APTrust both had a focus on performance and availability. My current project, by definition, must focus on availability and reliability.

After several months of working in Clojure, I noticed that the deeper I thought about the problem I was solving, the shorter my code became. Working in object-oriented languages, I’ve generally found the opposite to be true: the more you analyze a problem, the more classes you wind up with. This is particularly true in statically-typed object-oriented languages. Abstractions in Object-Oriented Languages In object-oriented languages, classes are supposed to be abstractions.

Ruby is a beautiful, intuitive and expressive language. It has a rich set of libraries, excellent support for packages and namespaces, and permits you to write clear, explicit code. These features make it suitable for large, complex applications. Unfortunately, Ruby on Rails employs a number of practices that complicate the process of developing and maintaining large-scale applications. Rails’ worst practices include a heavy reliance on assumptions and metaprogramming. In virtually any application, the code is the authoritative documentation.

I write programs from users. I write code for developers. Programs Well-designed programs typically have the following characteristics: They are intuitive to users. They represent concepts in ways that users already understand. They do not force users to learn new concepts to accomplish things they already know how to do. The gestures supported by Apple’s touch UI are an excellent example of intuitive design. People who have never seen an iPad before can start using one almost immediately.

I went to a presentation this evening from Rob Lee of Mandiant Corporation. He described what is known in information security circles as the Advanced Persistent Threat. This is a summary of what Mr. Lee discussed. When we think of hackers attacking a network, we tend to think of “smash and grab” attacks, in which the hackers gain access to a network, take everything they can scoop up, and try to get out as quickly as possible.