Oct 16

Endianess in Computers

Posted on by

The other month, I was working on some code where I had to be concerned about the endianess of the data.  I came across this article from IBM which greatly cleared up some things(I got to the article at least partially from this Stackoverflow quesetion).

Now I’m going to deviate here as that was almost a month ago and I forget what I was going to talk about here.

Anyway, the point here is that when you do any sort of bit-shift or bit-masking, it is always in the code as big-endian format.  When you do an operation, it is always big-endian but how it gets stored differs on what machine you are on.  To get the actual layout of the byte as they are in-memory, I generally cast the data as a uint8_t*.  So for example, let’s take the following code:

uint32_t value = 0x12345678;
uint8_t* value_memory = (uint8_t*)&value;

for( int x = 0; x < 4; x++ ){
    printf( "0x%02X ", value_memory[ x ] );
}

With this code, there will be two different outputs depending on what endianess your machine is in.  On a big-endian machine(like MIPS) this will print out something like the following:

0x12 0x34 0x56 0x78

However, on a little-endian machine(x86, ARM) it will print out the values in the opposite direction:

0x78 0x56 0x34 0x12

The thing to remember here is that the ‘value’ variable, when printed out on both big and little-endian systems, will print out as the same value.

If you need to force a particular byte order, make an array in the code(this is shown in the IBM article).  If you need to convert a particular byte sequence into the proper endianess for your processor, you can simply OR the two bytes together.

For example, let’s assume that we get the following two bytes in big-endian order:

uint8_t bytes[] = { 0x12, 0x34 };

To convert this to your processor endianess as a 16-bit value, you only have to OR the two bytes together, and since OR-ing will always act as though the bytes were in a big-endian format, your data has magically become the proper endianess.

uint16_t proper_endianess = (bytes[ 0 ] << 8 ) | bytes[ 1 ];

Anyway I hope that somewhat clears some things up.  (Although I think I may have simply confused people more).

Aug 20

Libraries and Dependencies

Posted on by

Have you ever used a library in a project, only to find that it depends on other libraries, only to depend on other libraries?

I’m willing to bet that you have.  Fortunately, there exist tools in the world to help manage these dependencies.  In the Java world, Maven is the de facto standard in this regard.  It’s quite convenient to manage your dependencies.  On Windows, NuGet is the de facto standard.  (Note: I have never used NuGet.  The number of dependencies can quickly get staggeringly huge though, so that is at least part of the reason that these tools exist.

Why do I bring this up? Well, at work we use jOOQ for our database operations.  It’s an awesome library that you should use if you need to do database operations in Java.  They also have a blog that I read from time to time, as there is some good information included in it.  One of their more recent blog posts talks about libraries and having zero dependencies.  Basically, the argument here is that all libraries should have zero dependencies, because of getting into a dependency hell(although they do carve out an exception for frameworks such as Spring and Java EE).

I don’t fully agree with this statement, but it is a good time to talk about these dependencies.  As far as I am able, none of the libraries that I have made have any dependencies(that being said, I only have one library on Maven: JavaSerial).

Why don’t I fully agree with the statement?  Well, in many cases I don’t think that it is entirely unreasonable to have a dependency, although this can get into a very gray area.  In the case of jOOQ, it would not be entirely unreasonable to have a dependency on commons-lang, because there are two different artifacts for commons-lang(one for version 2.x[commons-lang] and one for version 3.x[commons-lang3]).  On the other hand though, if there is only one class that you need from a certain dependency, it is reasonable to simply wrap that class as jOOQ does in order to provide the same level of support.

With that being said, here is a set of questions you should ask yourself when deciding if you need a hard dependency of a library:

  • What is the dependency attempting to provide?  For example, SLF4J is a reasonable dependency to have, since many libraries already use it for logging(although, if at all possible, you should use java.util.logging instead)
  • How much of the dependency do you need?  If you only need a single class from commons-lang as jOOQ does, pulling in the full dependency does not make sense.  If you use a significant portion, then it may make sense.  This is probably the most important question you have to ask.  If you are doing a lot of reflection, then it may be worth it to get a library to help you do reflection.  If you only need to do it once, it probably doesn’t make sense.
  • How widespread is the library?  Commons-lang is a widely used library, so having it as a dependency is not likely to cause problems.
  • Are there different versions of the library that people may be using?  As shown in the jOOQ blog post, if you are using Guava 15 while another library uses Guava 18, will that cause problems?  I have never used Guava, so I don’t know if it would actually cause a problem, depending on how the API has changed.

I realize that this is not a comprehensive list, nor is it a perfect guide.  Ultimately, as a library writer you should try as much as possible to keep the number of dependencies down as low as possible.  Otherwise, it can quickly lead to dependency hell for the end users, and potentially for you as well.

Jul 08

Debug Qt on remote device

Posted on by

Have you ever had to debug a Qt program running on a different computer?  There are a few special things that you must do in Qt Creator in order to properly get all of the debug information out from the device.

In my case, I’m debugging a Qt program running on an ARM processor, while debugging through my x86 computer.

First of all, you will need a root filesystem on your local computer that matches what is installed on your board.  See my guide on cross-compling Qt for ARM for information on how to use qemu-debootstrap to create a new rootfilesystem.  Alternatively, this should also work with any other filesystem that you create(e.g. through yocto).

Now that you have your root filesystem, here’s how to setup Qt Creator correctly:

  1. (optional) install the qt sources.  On Debian, you can easily do this with: 
    $ apt-get source qt5-default
  2. (optional) Once you have the Qt sources, go to Tools -> Options -> Debugger and click ‘Add Qt sources’ at the bottom and add the source path for Qt.  The source path should be /var/tmp/qt-src, the target path should be the directory in which you executed the apt-get source line above, for example it may be /home/username/qt/qtbase-opensource-src-5.3.2+dfsg/src
  3. Under Tools -> Options -> Debugger, go to the GDB tab and add these additional startup commands: 
    set sysroot /path/to/rootfs
set debug-file-directory /path/to/rootfs/usr/lib/debug
  4. Install the debug symbols in your rootfs: 
    $ sudo chroot /path/to/rootfs
# apt-get install qtbase5-dbg
  5. Go to Window -> Views -> Debugger log.  This will open up a new panel in Qt Creator that gives you the output from GDB.
  6. Run the program with GDB through Qt Creator as you would normally.
  7. When you hit a breakpoint or encounter an exception in the program, send the command ‘sharedlibrary’ to GDB.  This will load the shared libraries, but it will not update your stack trace immediately.  Either switch threads, or through the program for the stack trace to be updated in the GDB view.

You should now have a fully functioning remote debugging system.

Jun 12

Sequel Escalation

Posted on by

I’d like to take a moment to talk to you today about sequel escalation.  In general, this is when the next movie in a series has an even worse bad guy than the previous one.

(This is not limited to only movies though)

I was thinking about this as I watched the Fast and Furious franchise, although this also happens in the new Bond films.  I know that it is in someways inevitable; after all, if the new bad guy was easier to defeat, that would make for a bit of a dull movie, right?  But that is in some ways the problem.  There can only be so many international terrorists and criminal masterminds.  Once you get rid of one of them, a new one does not magically appear.

There has to be some kind of limit here.

But that never seems to happen.

Anyway, just some random thoughts.

Apr 25

About Script Kiddies

Posted on by

I don’t know anything about hacking.  It’s true.

I know the basic theory behind some of the attacks, but in-depth knowledge is beyond me.  I went to school for almost five years to learn how to program, and I can’t even tell you how to hack into a computer.

 

I like to think that the software that I write is immune to being hacked, but I don’t know for sure without having somebody else look for vulnerabilities.  At this point, if I were to try to hack something, I would be a script kiddie.  And that’s not something that you want to be.

At least there are a few guides on how to not become a script kiddie.

Apr 06

Setup Tomcat 8 on Debian 8 on port 80

Posted on by

Alright, that’s a confusing title.

Anyway, the point of this is to show you how to setup Tomcat 8 on Debian 8 (Jessie), and to have it bind to port 80.  This uses just the default packages from Debian, so if you are using a version of Tomcat that you downloaded directly from their website you may be out of luck.

First, install the needed packages(note: this causes Tomcat to start automatically):

$ apt-get install tomcat8 authbind

Now, we need to change the configuration settings for Tomcat and authbind to make sure that they work properly.

  1. Edit /etc/default/tomcat8.  At the very end, there should be a line that says AUTHBIND=no.  Change this to say AUTHBIND=yes
  2. Edit /etc/tomcat8/server.xml.  In this file, there is an XML tag that starts out with <Connector port=”8080″ ….  Change this to be 80.  There are two sections that you want to change this.
  3. Go to /etc/authbind/byport.  Do the following(note: this assumes that tomcat is running as the tomcat8 user, which it will do if you installed it through apt) 
    $ cd /etc/authbind/byport
$ touch 80
$ chown tomcat8 80
$ chmod 744 80
  4. Restart Tomcat 
    $ systemctl restart tomcat8

You should now be able to access Tomcat on port 80.

Mar 27

Inception dreams

Posted on by

I had an interesting dream the other night.  In my dream, I was explaining to a person a dream that I had had two nights before.  Also, for some reason, we were standing on a bed.

It seemed very Inception-like, talking about a dream within a dream.  Once I woke up, I was also a little confused about how I remembered a previous dream in another dream, as generally your dreams don’t stay with you when you wake up.

Anyway, that was kinda interesting.  Just a random thought.

Feb 15

Qt UDP Socket – specify source port

Posted on by

Have you ever had to use a QUdpSocket to send data from a specific port?  I had to do this earlier, and I figured out how to do it(although not using Qt-only APIs).

My problem was as follows: I need to be able to receive on a specific port.  The device that I am talking to listens on this port and then responds back on that same port, no matter what my source port is.  So to keep things clean, I figured that it would be best to use just one single socket to send and receive data packets.

The general consensus that I saw after searching Google was that it is not possible to do this.  It is possible, but it doesn’t seem possible using Qt-only APIs.  I knew that it was possible, as I have done this before.  This is the code that I came up with:

 

    int optval = 1;
    int fd;
    struct sockaddr_in socket_config;

    memset( &socket_config, 0, sizeof( struct sockaddr_in ) );

    fd = socket( AF_INET, SOCK_DGRAM, 0 );
    if( fd < 0 ){
        // error handling goes here
    }

    socket_config.sin_port = htons( port );
    if( bind( fd, (struct sockaddr*)&socket_config, sizeof( struct sockaddr_in ) ) < 0 ){
        // error handling goes here
    }

    if( setsockopt( fd, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof( optval ) ) < 0 ){ // error handling goes here } 

    udp_socket = new QUdpSocket(); 
    udp_socket->setSocketDescriptor( fd );

This allows us to still re-use the proper port, but use the socket in a Qt-like way.

Jan 22

Java RMI Example

Posted on by

So I just spent the last few hours attempting to get Java RMI working.  This was not easy.  However, the good news is that now you can benefit from it!  Link at the end, but first my story…

Last night, I was wondering how you would connect two Java applications together.  The reason for this is because I want to try and have an embedded Tomcat server inside of an application, but also have the capability of developing the webapp separately, since developing both the embedded server and the webapp in the same project did not seem to work very well.  I figured that I could use something like DBus-Java to do this, however I wanted it easily cross-platform.

As it turns out, Java already has something like that built-in!  In the form of RMI, or Remote Method Invocation.  However, the example that they give in the Java Tutorials is not particularly clear.  It also seems to be out of date, although if it actually is I am not sure.  There are several important parts that the tutorial does not go over.  Other documentation talking about what you need to do does not appear to be up to date either, much if it appears to date from Java 6 and before.

Since this was such a hard thing to figure out, I’ve made a standalone Maven project that goes and sets up an RMI client and server.  Hopefully it will help somebody else in the future.

Check out the code on GitHub!