blog

Difio Is Shutting Down on September 10

@ Aug 10th, 2014

Hey fellow developers, I have some bad news: Difio will be going away on September 10.

All paid subscriptions have already been canceled and the service will continue to operate normally until September 10. After that it will be turned off.

I’m very sorry for the disappointing news; I know a number of you loved the service!

If you have any questions, please get in touch with us!


Alexander Todorov, Lead Developer
@atodorov_

Difio Is Open Source

@ Apr 9th, 2014

Difio is pleased to announce that we’re now open source! As promised earlier the core service components and functionality have been made available under the Apache Software License.

Difio is a Django based application which keeps track of packages and tells you when they change. It provides multiple change analytics so you can make an informed decision when or what to upgrade.

Difio is relatively large application which contains multiple components. Explanation and example settings.py configuration has been provided in the README file. The default settings are made to work on a Linux/Unix system with minimum dependencies on external services. In a production environment you will likely want to change these settings - for example file storage server, messaging layer, email backend and caching layer to name a few.

The core difio/ application contains the logic for comparing package versions and producing analytics between them as seen on www.dif.io. It is missing the authentication backend(s) and user profile modules available in our SaaS offering. You will have to plug your own code here.

We encourage developers to download and install Difio on their systems and report bugs and issues to us. For development queries please use the difio-devel group.

Software Releases During 2013 as Seen by Difio

@ Jan 6th, 2014

Happy New Year everyone! Difio wishes you more bug-free upgrades and more frequent deployments during 2014. In this blog post we’re going to summarize the previous year as seen by our updates analytics service.

23% Newly Released Packages

Difio continuously monitors upstream package release feeds and notification hooks across six different programming languages. In 2013 there were 346914 new package versions of which 80135 are newly created packages, which didn’t exist before. The rest are new version releases to already existing software.

A fraction of these have been deployed by users of Difio which lead to a total of 13077 analytics processed by our service.

Release Activity

In 2013 three languages stand out as being most actively used to develop new software or improving existing packages:

  • Node.js - 143 thousand package versions;
  • Ruby - 102 thousand package versions;
  • Python - 61 thousand package versions.

The top 5 packages which produced the highest number of new versions during 2013 are

The most prolific months for developers turned out to be April, March and December, producing over 34 thousand releases each month. Some 6000 package versions were released between Christmas and New Year’s Eve alone!

Language Popularity

Developers tracking their applications at Difio prefer Ruby and Python, while Node.js comes third. The most frequently analyzed packages in each language are

  • Ruby - guard(155), newrelic_rpm(154), excon(151), sass(132), guard-rspec(127);
  • Python - raven(259), boto(190), celery(185), billiard(181), kombu(177). The famous Django framework comes sixth with 162 analytics processed by Difio;
  • Node.js - browserify(214), connect(132), uglify-js(125), request(99), express(80).

From all we can tell important packages like web frameworks are kept more consistent with respect to version usage while auxiliary packages and dependencies vary greatly across deployed versions.

Are we missing something? Let us know and we will do some coding magic to extract the information from our database. Happy New Year!

Introducing Amber, Travis and Beaker - Difio’s New Subscription Plans

@ Oct 22nd, 2013

Today Difio is pleased to announce our new subscription plans - Amber, Travis and Beaker.

Effective immediately all upgrade analytics will be accessible to everyone with a cut-off based on the package release date. If you upgrade your apps frequently this will keep bugs away and not cost you a dime. On the other hand the older the packages you use, the more you will have to pay to access the upgrade analytics.

What Subscription Plans Are Available

  • Amber - free - allows access to analytics for packages released in the last six months!
  • Travis - $ 1 per month - allows access to analytics for packages older than six months!
  • Beaker - $ 10 per month - allows access to analytics for packages older than one year!

The default subscription plan is Amber however you may upgrade or downgrade between Beaker and Travis as you see fit. More information about subscriptions is available at your profile page.

How It Works

Difio already color codes installed packages in App Details view. See our previous post about this. Subscription plans cut-off access based on the release date of the currently installed package.

"Amber"

As seen above, Amber plan only allows access to upgrade analytics for packages released in the last six months.

"Travis"

With the Travis plan access is granted to upgrade analytics for packages released in the past one year. Everything older than one year is disabled!

"Beaker"

Beaker plan allows access to everything. Compare with the image for Travis plan, this is the same application as seen under the two subscription plans.

Other Changes

There have been lots of minor bug fixes and improvements in the last couple of months. Most notable is the website optimization for mobile devices. You can now check your apps status on your smart phone as well.

What’s Next

As announced previously Difio will open source its code base. We have already started cleaning up and refactoring some internal code. Our plan is to have as much as possible of the code open until Christmas 2013 so stay tuned!

Latest Changes and Deprecated Functionality

@ Sep 10th, 2013

Hey everyone, Difio is always hard at work to improve our package tracking and updates analytics service. This is what we have been working on lately:

Redesigned Application Dashboard

We have redesigned the application dashboard which now shows icons based on your programming language and a text status for each app, telling you how many packages are outdated and how many are older than one year. Check the picture below or login to see yours.

"New Apps Dashboard"

Refreshed App Details View

The application details view has also been refreshed a bit! Non-essential fields were removed for less clutter and new color code has been activated. Installed packages are now colored in orange or red if they are older than six months or one year respectively. This is an indication that dependencies may be becoming stale and you have to plan your upgrade path soon.

"New App Details View"

Notification Email Changes

The daily and weekly notification digest emails were updated to specify applications affected by each particular update. This feature has been requested by our community of users and we were very happy to finally implement it!

To view your updates analytics you will have to login into your dashboard. Links to individual update analytics are no longer sent via email.

Follow Packages is Deprecated

Unfortunately the Follow Packages functionality had to go! A very few people were using it and where possible we have converted their records to individual applications.

Previously it was possible to follow a package without specifying any particular version of it. There were only 56 such objects in our database which were not converted.

What’s Next

Last month Difio has announced a road-map for the future. We’re going to continue working on this and bring you an easier and smoother application import next time.

In the mean time, don’t forget to check your updates and upgrade your apps!

Important Changes to Difio - Registration, Subscriptions and Open Source

@ Aug 21st, 2013

As Difio’s Lead Developer and on behalf of the Difio team, it is my pleasure to announce today our future plans for Difio. This is a high level outline of pending changes which will be implemented in the upcoming weeks. Each of them will be announced in more details later.

Easier App Registration

Currently there are several ways to register applications into Difio, both manual and automated and several popular PaaS providers are supported. Automated registration is done via script, which needs to be installed and configured into your environment. This proved to be confusing and hard to use.

We’re changing the registration model to read your app dependencies from a publicly accessible URL containing your requirements.txt, Gemfile, etc. You will have the flexibility to create your own script to refresh this file contents, no more installing extra packages from us.

Cloud related registration methods will be updated as well and refactored as plugins modeled after the cloud vendor’s specifications.

Existing app registrations will be migrated to work appropriately.

Together with this feature Difio will introduce much wanted from everyone integration with GitHub and BitBucket.

Changes To Subscription Model

The present subscription model requires payment if you’d like to access additional upgrade analytics. This will change. Analytics tests will be accessible free of charge!

The paid subscription will cover access to analytics for older packages. After all Difio is here to help your apps stay up-to-date. We’re currently thinking about a one year cut-off based on the package release date.

Open Source Here We Come

Everyone involved with Difio (both users and developers) has their roots in the open source movement, myself included. We’re going to open source as much as possible of our platform!

At first all analytics tests will be refactored and made open. Our goal is this to coincide or closely follow the changes of the subscription model and have both open source and free of charge analytics.

Following this event other parts of the platform will be open sourced gradually where appropriate.

Other Minor Changes

Since the announced changes will touch base with lots of Difio’s internal components we will use the opportunity to refresh some extra code as well like the user interface and the email notification sub-system. All changes will be announced in our newsletter.

I can’t wait to start writing code and transforming the platform in response to your great feedback. In the mean time enjoy Difio and keep your apps up-to-date!


Alexander Todorov, Lead Developer
@atodorov_

Difio Analytics Tests

@ May 7th, 2013

Today Difio is pleased to announce the general availability of multiple tests which are designed to test and verify potential issues or known bugs.

What tests are available

  • API diff - Difio generates API definition from the package source code. Changes in API could be backward incompatible and result in application crash and runtime failures. API diff is currently available for Python, PHP and Java.

  • Full diff - this is the full git diff between two package versions. It is available for informational purposes.

  • Package Size Change - packages that change their size significantly can be a sign of a corrupted build. This test reports any size changes over 30%. See DIFIO-12243 for example.

  • File Size Change - inspects files which are present in both versions. Any size change that is both more than 20% and more than 20 KB or any file that changes from being zero-sized to non-zero sized, and vice-versa is reported.

  • Added non-text Files - new files are classified with libmagic. Any non-text file is reported. Bugs like Django #19858 can be found with this test. For example see DIFIO-12987.

  • Modified Files - lists all modified files. You should manually review the list and inspect if unwanted files were modified.

  • Added Files - lists all newly added files. New files are usually resources, tests of new functionality. Manually inspect to verify additions are expected.

  • Removed Files - files in packages should be removed only if properly obsoleted or initial inclusion was a bug. Removing modules or parts of them can break the API.

  • Renamed Files - renaming files can break API. This test lists all renames between two versions.

  • Permissions Change - file permissions change (Linux/UNIX only) could indicate security issues. In a source package without executable programs permissions should not change.

  • Symlinks - this test traverses the entire package tree and reports any symlinks found. It will report FAIL if any symlinks exist in the new package. For example see DIFIO-16005.

  • File Types Change - this test compares file types reported by libmagic between two package versions. Files should not be changing their type between updates.

  • Virus Scan - a virus scan is performed on the package using the ClamAV virus scanner. Even though it is unlikely that packages contain viruses, it’s still important to know if anti virus tools will trigger false positives.

  • Test Cases - this test reports any changes in test cases. It counts the number of available test case files in some well known locations. If new version is missing tests or has less then severity will be VERIFY or FAIL.

Subscribe Now!


Difio is constantly working on improving the service. In the mean time start following packages, invite your friends and send us your feedback via GitHub or Twitter.

API Diff Support for Java and PHP

@ Mar 10th, 2013

Today Difio is pleased to announce API diff support for Java and PHP programming languages.

The API diff feature extracts class and method signatures and compares the results between an older and a newer version. If API has changed this is a good indicator that there were substantial changes in the new package version, which require through testing.

With this release API diff becomes available for source code written in Java, PHP and Python. We are looking to support other languages as well.

What’s next

Difio is constantly working on improving the service. In the mean time start following packages, invite your friends and send us your feedback via GitHub or Twitter.

Improved Support for PHP

@ Mar 6th, 2013

Today Difio is pleased to announce improved support for the PHP programming language and the Composer dependency management tool.

Previously only PHP PEAR packages were supported and it was not possible to manually import a list of packages. Starting today, these issues have been fixed.

At the moment Difio supports three upstream sources for PHP packages:

Users are now able to manually import package list from their PHP applications using Composer.

To register PHP apps go to https://difio-otb.rhcloud.com/application/import/composer-show/? and paste the output of php composer.phar show --installed command.

For example:

$ ./composer.phar show --installed
doctrine/common                              2.3.0   Common Library for Doctrine projects
imagine/Imagine                              v0.4.1  Image processing for PHP 5.3
pear-pear.php.net/HTTP_Request2              2.1.1   PHP5 rewrite of HTTP_Request package ...
pear-pear.php.net/XML_Util                   1.2.1   Selection of methods that are often needed when working with XML documents ...
pear-pear2.php.net/PEAR2_Cache_SHM           0.1.0   Allows you to share data across requests as long as the PHP process is running. ...
pear-pear2.php.net/PEAR2_Text_Markdown       0.1.0   This is a port of Solar_Markdown.
psr/log                                      1.0.0   Common interface for logging libraries
symfony/symfony                              v2.2.0  The Symfony PHP framework
twig/twig                                    v1.12.2 Twig, the flexible, fast, and secure template language for PHP

This application contains packages from the three different sources and they will be properly parsed and recognized by Difio.

What’s next

Difio is constantly working on improving the service. We are now looking into API diff support for PHP among other things, so stay tuned! In the mean time start following packages, invite your friends and send us your feedback via GitHub or Twitter.

New Dashboard and Follow Packages Functionality

@ Feb 25th, 2013

Today Difio is pleased to announce our new dashboard and the ability to follow packages without registering applications. Read below for more information.

Latest Releases

Latest Releases

The default dashboard page is showing a news stream of all latest releases for packages the user is following or using inside applications.

Change History

Change History

This page is showing a news stream of change analysis between multiple package versions that the user is following or using inside applications. As shown in this example the user is following bundler-1.2.1 and bundler-1.2.3 while the latest available is bundler-1.3.0 and bundler-1.2.5.

Follow Packages

Follow Packages

A new page to view and search packages is now available. Users can follow or unfollow any packages and versions they want through the buttons. Simple color codes distinguish between different states: green - user is following version; gray - version is used inside registered application; transparent - not following version. Versions which are used inside applications can only be removed/unfollowed by deleting the entire application, or deleting the package from the application. For example Pygments-1.5 is been used in an application, but Pygments-1.6 is been followed manually.

It is highly recommended to follow a particular version, not only the package name. If a user decides to follow only package name, but not versions, Difio will not be able to generate change analysis reports because there’s no basis to compare to. In this case the user will be able to see only the latest releases news stream.

Applications

This page is showing all registered applications for which Difio is monitoring dependencies. This page has been available previously and also includes the improvements announced last week. Small speed improvements have been made to this page as well.

Invite Friends

This is an interactive dialog which allows you to invite your fellows to try Difio. It only asks for the email addresses of your friends and sends them an invitation. We do not keep track of these addresses nor send any spam to them.


We encourage developers from all supported backgrounds and languages to start following packages and send us their feedback via GitHub or Twitter.