SOFTWARE ENGINEERING blog & .lessons_learned
manuel aldana
Manuel Aldana

>January 26th, 2012 · 1 Comment

The evil path of Mobile-Apps (vs. Webstandards)…

The so called Apps are the usual End-User applications running locally on Smartphones and Tablets (similar to Desktop applications). From usability point and hipness factor they offer great moments. But as a Software Developer I am extremely skeptical… Well, what is wrong with all these Apps?

Step backwards from Web-Standards

I think web-standards like HTML, JavaScript, CSS are key technologies for the Internet. The Webapp-Provider can develop in a very agile way as the application is released on servers and ready to be accessed by Webbrowser. The User on the other hand doesn’t need to install anything, the only “application-gate” is the Webbrowser, simply try out a website, toss it away or revisit. No installation/removal necessary. The URL is your application-hook.
On the Mobile App side big step backwards: I now face the pain that every website has its own app which basically doubles content. I need to go through the annoying search/install/upgrade/removal path to try out things. Instead of keeping simple URL-Bookmarks for several websites the Desktop now is packed with tons of distracting App-Icons and alarming Upgrading Notifications.

High Implementation Effort

On top of your HTML/Javascript based Webapplication and optional HTTP API you need to add effort to more apps. Your whole application setup gets fragmented and increases maintenance and developing efforts to a big extent. Today 2 major platforms exist (Android, iOS) and it seems that Windows Mobile platform will follow. Yes, on the Webapplication side you indirectly also need to support multiple Browsers (Safara, FF, IE, Chrome etc.) but this effort is much less as maintaining desktop-lik Mobile Apps.

App-Store Drawbacks

Smartphone Apps are typically distributed through App-Stores (like Apple’s App-Store or Google’s Android Market). These App-Stores and their review process do have their pros: They act as a single entry point for end-users, which can search/browse and rate apps which is really comfortable. Also the review-process and policies can kind of enforce a style guide which can positively influence usabilty. Besides malicious apps are easier to filter and kick-out from App-Store. On top it seems that users tend to be more willing to pay for certain apps in contrast to pay for a website content, which is good for the App-Providers.

Never the less there are a lot of disadvantages which makes Software Development for Mobile-Apps tough:

Missing/Intransparent Auto-Upgrade

It can happen that your current installed app is inconsistent to the latest released one (I see this through circle icon top-left of app or App-Store icon) and isn’t upgraded on the fly. This causes headaches both on user and App-Provider side. The user is annoyed because he/she need to actively upgrade App-versions all the time and at some point will simply not do it. The App-Provider has to invest a lot of developing and testing resources to make all the backend (like APIs) compatible to very old App-releases as there is no guarantee that very old Apps aren’t “out there” any longer. Down compatiblity and supporting multiple application versions is in my view one of the biggest cost drivers in Software Industry.

Inefficient Packaging

When upgrading an App the whole binary package needs to be downloaded. This is extremely inefficient as usually between releases only a smaller part of App changes (source-code diff). Especially in areas where bandwith isn’t good downloading a 5MB upgrade is a big pain. A more sophisticated packaging build tool which makes binary diffs possible would ease a lot and come hand in hand with an Auto-Upgrade feature. To me it is a mistery why none of current App-Stores have such a feature…

Release/Rollout delay

Due to the app-review process there is a delay between your internal approval of app and the final downloadable app inside App-Store. This usual takes a week but also can take longer (e.g. during peak times when many app-providers are offering new apps or versions at the same time). Thinking of being agile and releasing software often (see Continous Delivery) this is a major drawback. You have to invest much more effort on testing your app-package as major production issues could cause an app being unusable without possibilty to react quickly as major bug fix releases need to go through review process again. In such an error-risk environment changes are done much more defensively and you also will meet all the other disadvantages of NOT RELEASING OFTEN.

Distribution dependency

App-Store owners (Apple, Google) have full control whether an app is available or not. Apple even has policies disallowing apps which are implementing similar functions as preinstalled Apple ones (Email-Client, Webbrowser). You are dependent on the good will on the reviewers. These hard restriction aren’t found for typical Desktop or webapplications. In Desktop case you simply directly a bundled application-package. As webapplication provider you simply rollout you app and let users find you by the “application”-gate the Webbrowser.

Future-Hope

My bet and hope (esp. as a Software Developer) is the HTML 5 standard + Responsive-Web-Design movement. It keeps the highly flexible Webapplication oriented approach and offers a single point for both Desktop Webbrowsers and Mobile devices (frontend is “adapting” to end-device). My gut-feeling is undermined as big web-player Google also seems to go this way, e.g. there are hardly any dedicated Google Mobile Apps rather they try to tackle the Mobile Usability problem directly on the Webapplication side.

→ 1 CommentTags: Software Engineering · Technologies/Tools · Uncategorized

>September 30th, 2011 · No Comments

Complexity drivers of Software-Systems

More complex software-systems correlate with higher lead-time (time-to-market of initial idea to user-available software) and fragility. They also tend to have negative influence on usability. Therefore it must be a goal to reduce following complexity factors to a lowest possible degree.

Codebase size

Independant of what the codebase does it incorporates maintenance-efforts: On big codebases it takes longer to implement changes, more code needs to be read/comprehended. Codebase size also correlates with longer builds, deployment processes and startup times, which means a latency-increase of feedback loops. Encapsulation/Modularizaton can be applied, but it only weakens the impact: You won’t be able to reduce efforts of a 100M codebase compared to a 10K one by just having good code-quality.

Codebase quality

Decent code-quality is important for implementing changes quickly (readable code, separation of concerns, DRY etc.). Good quality also includes test-coverage so regression-bugs are better caught and there is a safety-net during refactorings. Though business often neglects less visibile code-quality, it is essential to reduce lead-time. In severe cases you’re at a dead end: code is such in a bad shape, that it is impossible to evolve (changes are too side-effect-risky and/or too expensive).

Tools/technology diversity

Itself diversity of tools (programming-languages, frameworks, hardware etc.) is good because you can choose the right one to solve your specific problem. But it also increases risk: single-person-know-how, legacy-tech, learning-curves, beta/buggy-stability, upgrade/patching-efforts, transitive dependencies (see .dll, .jar nightmares).

Integration points

Though distributed systems are necessary (scaleability, reliability, modularization, partner-integrations) they are more fragile: Monitoring and deployment efforts increase and security breaches are more likely. Also tracing, debugging and testing efforts are higher.

Organization Size

One of the biggest factors is the size of your organization because efforts increase squarely (see also Mythical Man Month). Bigger organizations try to fight this by introducing hierachies and heavy-weight processes, but this structure can have negative impact on “short/quick” decisions and slows down speed. In some scenarios political-games emerge, which block progress considerably.

External partners

External-partner-integrations (most likely over APIs, batch processing) need more investment: Different release cycles need to be taken into account, compatibility must be offered and dedicated monitoring needs to be setup. Also communication is less direct (fixed calls, meetings, travelling).

User-Base + Traffic

A bigger User-Base means you have to spend more effort on support. Because of the mass statistically more edge-cases come up and need to be handled by software. Also scalability requirements need to be implemented by more sophisticated production environments. Popular systems also attract criminal activity, which you need to respond with higher security-investments, which again make the system less usable. Big applications also produce more data, which needs to be maintained (compatibility, migrations, analyzation).

Being Complexity aware

Above factors cannot be reduced to Zero (you will always meet an Inherent Complexity), but you should fight back:

  • Prioritize, prioritize, prioritize: Featuritis has bad impact on usability and codebase-size. Implement important features only. Remove unneeded features and cleanup code.
  • Consolidate technologies: Don’t introduce a new technology just because it was praised in the last magazine. Evaluate it and maybe use it privately first.
  • Take Fowler’s advice serious: “Don’t distribute, if you don’t have to”
  • Don’t always increase team-size, rather have a small team with A-players.
  • Quantify: Use metrics for business success, code-quality and production stability. Especially for bigger systems gut-feeling isn’t enough.

→ No CommentsTags: Software Engineering · Software Maintenance

>July 3rd, 2011 · No Comments

Accessing rrdtool-files Data with Java / Scala

Though rrdtool is widely used especially for monitoring, it took me a while to find a simple and compatible bridge for reading data of rrd-files with Java. Following hopefully helps others to save some time and to workaround some pitfalls.

My requirement was:

  • Monitoring website showing trends of metrics, i.e. compare current with past values, both absolute and relative (hour over hour, day over day, etc.).
  • Data backed is round-robin-database. The rrd-files are generated by munin (with under the hood rrdtool). Data should be accessed read-only.
  • Website’s technology stack is Java based (Play! framework with Scala integration).

Because I use Play! framework I needed a way to access rrd-files with Java-Technology. My first look at rrd4j looked promising but failed, because rrd4j is a port and cannot read rrd-files created by original rrdtool. After some time-consuming research and Google-digging I finally stepped over java-rrd.

Installation Steps java-rrd

I couldn’t find the library built and distributed in any Maven-repository, so you have to download + build yourself:

# download + unpack
wget http://oss.stamfest.net/java-rrd-hg/archive/tip.tar.gz
tar xfz <downloaded-tarball.tar.gz>
# build .jar library with Maven
cd <untarred-directory>
mvn package
cp target/*.jar <your-target-lib-folder>

Instead of dump-copying and to be more clean with your build-system you might want to deploy the .jar file to a repository like Nexus.

Read-Access Usage java-rrd

As I use Play! framework with Scala integration, Scala was the integration way:

import net.stamfest.rrd._
….
val rrd = new RRDp(“/tmp”, “55555″)
val command = Array(“fetch”, “your-rrd-file.rrd”, “MAX”, “-r”, “1800″, “-s”, “-1d”)
val result = rrd.command(command)

if (!result.ok)
  println(result.error)
else
  println(result.output)

For completeness another snippet in Java source language:

import net.stamfest.rrd.CommandResult;
import net.stamfest.rrd.RRDp;
….
RRDp rrd = new RRDp(“/tmp”, “55555″);
String[] command = {“fetch”, “your-rrd-file.rrd”, “MAX”, “-r”, “1800″, “-s”, “-1d”};
CommandResult result = rrd.command(command);

if (!result.ok)
    System.out.println(result.error);
else
    System.out.println(result.output);

With java-rrd you can also access rrdtool over sockets/network.

Parsing rrdtool output

rrdtool output after fetch is plain text, something like:

        speed


920804700: nan
920805000: 4.0000000000e+02
920805300: 2.0000000000e+03
920805600: 0.0000000000e+00
920806800: 3.3333333333e+01

Above needs to be processed further to be “structured enough” for your code. For example do following (I only show Scala, I skipped Java iterating syntax hell…, did I mention that I love passing functions ;):

def outputToNumberPairs(rrdFetchOutput: String) = {
    // filtering unneeded + empty lines
    val list = rrdFetchOutput.trim().split(\n).filter((n) => n.contains(“:”) && !n.contains(“nan”))
    // parsing strings to numeric values and combine to pair
    for (i <- list) yield i.split(“:”)(0).trim().toLong -> i.split(“:”)(1).trim().toFloat.round
  }
GIVES BACK:
===
res34: Array[(Long, Int)] = Array((920805000,400), (920805300,2000), (920805600,0), (920806800,33))

→ No CommentsTags: Technologies/Tools