Informatics

One major difference when using Windows vs. Linux is that Windows "prefers" \r\n (CRLF) for newlines. This has been a known fact for years, and most tools will automatically handle this  (e.g., Filezilla). However, docker on Windows does not handle this, which will cause massive issues that are literally impossible to debug. For instance, you might get this error:

standard_init_linux.go:175: exec user process caused "no such file or directory"

This issue is really easy to fix: Convert your files from \r\n (CRLF) to just \n (LF). Most editors/IDEs on Windows can do that for you (including Notepad++). However, this will not be a permanent solution. If you are using GIT, you need to configure either your repository or your global GIT configuration to prefer \n over \r\n:

https://help.github.com/articles/dealing-with-line-endings/

Unless you really need to keep your \r\n, I recommend to change this globally with

git config --global core.autocrlf input

 

I needed a quick way of converting videos to gifs on Linux -ffmpeg and this post on stackexchange to the rescue!

The result is this neat little bash script:

#!/bin/bash

if [[ $# -eq 0 ]] ; then
 echo "Usage: videoToGif inputFile [outputFile] [FPS] [WIDTH]"
 echo "Example: videoToGif inputFile.ext outputFile.gif 60 360"
 exit 0
fi

inputFile=$1
outputFile=${2:-output.gif}

FPS=${3:-30}
WIDTH=${4:-360}

#Generate palette for better quality
ffmpeg -i $inputFile -vf fps=$FPS,scale=$WIDTH:-1:flags=lanczos,palettegen tmp_palette.png

#Generate gif using palette
ffmpeg -i $inputFile -i tmp_palette.png -loop 0 -filter_complex "fps=$FPS,scale=$WIDTH:-1:flags=lanczos[x];[x][1:v]paletteuse" $outputFile

rm tmp_palette.png

I used it to convert a Screencast created with "recordMyDesktop" (from OGV format) to a GIF.

During my recent journeys I discovered AngularJS, a JavaScript framework dedicated for writing Single Page Applications (SPA). One of the key concepts of Angular is the so called digest cycle. Everytime a user clicks a button, a timer caused by $timeout or $interval is fired, a $http request finishes or a $promise is resolved (or $q.defer is resolved), a digest cycle is executed.

This digest cycles causes the whole page to re-evaluate. All watchers would automatically check whether or not the variable (or function) they are watching have changed (in fact, this check is executed twice).

Now in some of my AngularJS Projects I had to implement a ticking clock, showing Hours, Minutes and Seconds. The most trivial way implementing this in Angular would be something like this:

$scope.updateClock = function() {
    $scope.curDate = new Date();
}
$interval($scope.updateClock, 1000); // update once per second
<p>{{ $scope.curDate | date:'hh:mm:ss' }}</p>

These lines of code are probably found in many AngularJS projects across the world. But what if I told you that this is in fact not a good solution? Here is why: $interval will cause a digest cycle! In this very simple example, it will cause one digest cycle per second. If you have some complex computation or many watchers, this could significantly slow down your SPA and lead to users abandoning your page. In addition, mobile devices will drain their battery much faster. The main question however is: Why would you want your whole application to evaluate again, when you only want to refresh time on your clock?

To fix this problem, we can help ourselfs by working  the concepts that JavaScript and HTML5 provide, and therefore make clocks  great again! To overcome the issue of causing unneccessary digest cycles we need to avoid using the concept of watchers and data binding of AngularJS. The experienced AngularJS programmer will already know what is coming next...

Make DOM manipulation great again

But but but... You aren't supposed to do that in Angular!

Right! You aren't supposed to do that in your controllers. However, you are allowed to do that in directives. Essentially, I am going to show you how to build an Angular directive which uses JavaScripts own setInterval (Note: You could use $interval and invokeApply=false) to modify a DOM element, displaying the current time.

    angular.module('angular-ticking-clock', []).directive('tickingClock', ['$filter', function($filter) {
        return {
            restrict: 'E',
            link: function(scope, element, attrs) {
                var updateTimer = undefined;

                var updateDateTime = function() {
                    element.text($filter('date')(new Date(), attrs.dateTimeFormat));
                };
                
                updateTimer = setInterval(updateDateTime, attrs.updateInterval);
      
                /**
                 * On Destroy of this directive, we need to cancel the timer
                */
                scope.$on(
                    "$destroy",
                    function( event ) {
                        if (updateTimer)
                            clearInterval(updateTimer);
                    }
                );

            }
        };
    }]);

The most improtant part of this is the $scope.$on("$destroy", ...)! Whenever this directive is destroyed, we need to clear the interval timer, such that it no longer fires. The second most important part is that this directive should always be used as an element (restrict: 'E'), ensuring that it has its very own DOM element to modify.

Other then that, that's it. Feel free to use this code as you like. I also created a github Repo and an NPM package for it.

While people on the Internet are still fighting about npm vs bower, there are some packages that are only available for npm and some that are only available for bower. Unfortunately you will run into problems, sooner or later, just like I did today.

The package pdfmake enables JavaScript applications to convert text to PDF, both within a website as well as a NodeJS server application. However, they explicitly state that the bower version should be used for web applications, and the npm version for server applications.

But if you want to use npm and want to avoid bower (for whatever reason), then you will run into a problem.
Thankfully, somebody created a wrapper package for npm: https://github.com/AaronBuxbaum/pdfmake-client

You can install it via

npm install pdfmake-client --save

2 Comments

One of the tasks I come across often is converting images from one format into another. For instance, I need to convert SVG to PNG.

This can be achieved easily by using the "convert" commandline tool (ImageMagick) and a standard for loop in linux (note that I wrote the *.svg statement in the for command on purpose and that I use "$f" on purpose):

for f in *.svg ; do
    convert "$f" "$f.png"
done

However, this produces ugly file names like "file1.svg.png", which could be desireable in some scenarios, but not in my case when I deploy it for a website. You can bypass this by using ${f%svg}png:

for f in *.svg ; do
    convert "$f" "${f%svg}png"
done

Essentially this tool can handle a lot of use cases, for instance you can specify the picture density and which color should be used as transparent:

for f in *.svg ; do
    convert "$f" -density 300 -transparent white "${f%svg}png"
done

If you only want your pictures to have a certain size, you can resize them, e.g. using -resize 64x64:

for f in *.svg ; do
    convert "$f" -resize 64x64 -density 300 -transparent white "${f%svg}png"
done

However, you need to be careful when doing this. For instance, when the source image is smaller than the destination image, you might run into problems, and need to use the command like this:

for f in *.svg ; do
    convert  -resize 64x64 -density 300  "$f" -resize 64x64 -density 300 -transparent white "${f%svg}png"
done

I recently came across some very odd behaviour in MySQL with German Umlauts (ö ä ü ß) and Unique Constraints. The problem is even documented as Bug #57860 on mysql.com. In short, MySQL (or rather utf8_unicode_ci) would suggest that foobär is the same as foobar. So the statement
INSERT INTO test (test) VALUES ('foobar'),('foobär');
where test is a column with a unique index/constraint would fail.

This behaviour might be desired in some languages, but particularly for the german language this behaviour is not optimal. I'm sure if you find this blog post, you came across the same problem so I do not need to come up with another example.

The solution is simple, though you should think twice before you use it:
Instead of utf8_unicode_ci, you could use utf8_general_ci or even the newer and more appropriate utf8_german2_ci (available starting with MySQL 5.6).

A Comparative Study of DASH Representation Sets Using Real User Characteristics
C. Kreuzberger, B. Rainer, H. Hellwagner, L. Toni and P. Frossard
Published at NOSSDAV 2016 (co-located with MMSyS 2016), Klagenfurt, Austria, May 2016.

Link to Paper: PDF
BibTeX Cite: To Appear
Presentation: PDF

Paper Website: http://concert.itec.aau.at/NOSSDAV_2016/
Git Repo: https://github.com/ChristianKreuzberger/amust-ndnSIM

Due to the new release of ndnSIM (version 2.1) my last post about running ndnSIM without root has become obsolete. It is also no longer necessary to compile ndn-cxx as a separate library now. However, if you still want to use the (recommended) ndnSIM scenario template without having root access, here are the steps to follow (for version 2.1):

Step 1: follow the installation (requirements, etc...) instructions on the ndnSIM website until you have to type ./waf the first time.

For instance (after having installed all pre-requesits):
mkdir ndnSIM
cd ndnSIM
git clone https://github.com/named-data-ndnSIM/ns-3-dev.git ns-3
git clone https://github.com/named-data-ndnSIM/pybindgen.git pybindgen
git clone --recursive https://github.com/named-data-ndnSIM/ndnSIM.git ns-3/src/ndnSIM

Step 1.a: If you require BRITE, do this in addition:

hg clone http://code.nsnam.org/BRITE
cd BRITE
make
export BRITE_HOME=$(pwd)
cd ..

Step 2: Create a directory where ns-3 and ndnSIM will be "installed" into, e.g.:
mkdir ndnSIM-build

Step 3: Go to the ns-3 subfolder and compile ns-3 and ndnSIM:
cd ns-3
./waf configure --prefix ../ndnSIM-build -d optimized
./waf

Note: --prefix ../ndnSIM-build tells the build-script to not install the libraries to the default location, but to ../ndnSIM-build.

Step 3.a: If you followed Step 1.a for BRITE, you will have to add --with-brite=$BRITE_HOME to the ./waf command:
./waf configure --prefix ../ndnSIM-build -d optimized --with-brite=$BRITE_HOME

Step 4: Grab a coffee, tea, beer, etc.! This step takes some time...

Step 5: Once this has finished, type
./waf install

Note: You did not have to use sudo! ns3 and ndnSIM are now being "installed" to ../ndnSIM-build

Step 6: Set up LD_LIBRARY_PATH and PKG_CONFIG_PATH for being able to use the scenario template
cd ..
export LD_LIBRARY_PATH=$(pwd)/ndnSIM-build/lib/
export PKG_CONFIG_PATH=$LD_LIBRARY_PATH/pkgconfig

Step 6.a: You might need to add those exports to your ~/.bashrc file.

Step 7: Download and configure scenario template
git clone https://github.com/named-data-ndnSIM/scenario-template.git scenario
cd scenario
./waf configure

Step 8: Create your examples in the scenario template and run them!

When using waf / wscript for compiling a project, you might come across the problem of adding another library, e.g., a shared object, to the compile and run process of waf. The task is trivial, but tutorials on that matter seem to be rare.

In my specific example, the task at hand was to add an external library, which I built from source, to an existing complex project. For now, we call this library libExternalStuff.
I assume that libExternalStuff has been built externally with whatever tools it was required to built, and it generated the following:
$SOMEDIR/libExternalStuff/includes/*.h (in particular, we are going to use externalStuff.h)
$SOMEDIR/libExternalStuff/bin/libExternalStuff.so
Where $SOMEDIR could be anywhere, e.g., in your home directory.

If you would "install" the library to your includes and library paths on Linux, you would probably be able to skip the following steps. Though if you don't want to mess with your Linux distributions configuration, or if you want to have a custom version of a certain library, this should be helpful.

Open the wscript file of your project (or subproject). Find the part where it says
def configure(conf):

Within this method, add the following:

test_code = '''
#include "externalStuff.h"

int main()
{
return 0;
}
'''

conf.env.append_value('INCLUDES', os.path.abspath(os.path.join("$SOMEDIR/libExternalStuff/includes/", ".")))

conf.check(args=["--cflags", "--libs"], fragment=test_code, package='libExternalStuff', lib='ExternalStuff', mandatory=True, define_name='EXTERNAL_STUFF',
uselib_store='EXTERNAL_STUFF',libpath=os.path.abspath(os.path.join("$SOMEDIR/libExternalStuff/bin/", ".")))

conf.env.append_value('PROJECT_MODULE_PATH',os.path.abspath(os.path.join("$SOMEDIR/libExternalStuff/bin/", ".")))

This should get you going when you call:
./waf configure

What are those lines doing? First of all, we are creating a test code, which tries including one of the generated/provided header files. This can be adapted to anything you would like, and is mainly a sanity check. Second, we are adding the includes directory to the global INCLUDES path. This is important, else the compiler wouldn't know where to find the include script. Then we are creating a configuration entry for libdash. I'm not an expert with waf/wscript, but this line seems to have worked fine for me. Most importantly, make sure to have the lib parameter set properly (e.g., if your shared object is called libExternalStuff.so, then the lib parameter reflects the -l parameter of gcc, and needs to be set to ExternalStuff). libpath is used for the linker to determine where to find the .so file.
The last line
conf.env.append_value('PROJECT_MODULE_PATH',os.path.abspath(os.path.join("$SOMEDIR/libExternalStuff/bin/", ".")))
is something specific to your project and you would have to figure out how the global variable is called in your project. This is the path where the project will look for the shared object file when using ./waf --run.

If your projct also has a --run method, then you will have to add the library to the run-configuration.
Find out where dynamic libraries are specified in your wscript, and add it to the line, e.g.:
module.uselib = 'LIB1 LIB2 EXTERNAL_STUFF'

I'm not an expert with waf/wscript, but those lines should get you going. However, based on the used project, some lines might have to be added, modified or deleted.