Posts

Project Stage 2: Benchmarks

     For stage 2 of the project I built my package on the systems I plan to test on and then ran a standardized test a number of times to get a benchmark for performance that I can use to measure the impact I have with my code changes. Since I am going to be testing on both x86_64 and Aarch64 I needed to mimic the build process I followed in stage 1 to have ssdeep on all required systems. First up was Aarchie, after using wget to retrieve the tarball from ssdeep's website I used tar -xvf to unpack it into my project directory. From there a simple ./configure followed by make -j4 (-j4 to allow 4 concurrent jobs to run) resulted in a complete build with the ssdeep executable ready to test. My smoke test was simply calling ./ssdeep on the tarball and I was returned: ssdeep,1.1--blocksize:hash:hash,filename 12288:IvvRytd0lLS4iXBpfBof3msXd2B1mOfw68Tsd:IvZvlLS40n02st2/mOt8od,"/home/cmcmanus/proj/ssdeep-2.14.1.tar.gz"      Thus my Aarchie build passed my smok...

Project Stage 1: Planning

     For the first stage of my final project I had to select a hash, checksum or compression package to try to optimize. The package also had to compile down to machine code, like C, C++ or Assembler. Each student needed to select a different package to tackle. To keep as many possibilities for optimization open as possible, I decided to look both on Aarchie and Xerxes for packages available on both.      I began my search by dnf searching for hash checksum and compression which gave me a ton of results. After sifting through these I first thought to focus on zpaq but discovered it's archiving and journaling processes were pretty complex and difficult to read through so I moved on. Next I was going to work with spooky hash but after building and profiling it I discovered one of my classmates had beaten me to the punch.      Thus I landed on ssdeep, a program used to take input files and create a context triggered piecewise hash, or fuzzy ...

Lab 3: Compiled C Code

Image
     In this lab I investigated the transformation of my code from source C code to C compiler output to learn exactly what the compiler is doing when I call it. My initial program was a simple Hello World program: #include <stdio> int main(){         printf("Hello World!"); }      My first compilation was a simple gcc with most of the optimization and fancy tricks stripped out. This was done with the -O0 and -fno-builtin flags, -O0 meaning do not use any optimizations when compiling and -fno-builtin meaning do use any built in function optimizations. So : gcc -g -O0 -fno-builtin -o hello1 hello.c.       The compiler then returns an Executable and Linkable Format file (ELF for short) named hello1 which I used to examine what it had done to my code. The first thing I noticed after using objdump to look at the assembly code that was output was that my simple four line program had ballooned out to...

Lab2B: Building glibc

      For part B of Lab 2 I was tasked with building my own version of glibc and then introducing a bug into it to ensure I am using my build instead of the installed version. Glibc is GNU's implementation on the standard C Library which underpins many functions of the Linux kernel, so it is very important that my build and tests don't affect the systems version of glibc or my classmates will not be happy with me.      The first step is to get the source code for glibc. Since I used wget and tar for part A, I will use git this time to get a feel for both methods. So running git clone git://sourceware.org/git/glibc.git  gave me the source code to build from but before I did there is an important step for glibc which is to create a separate directory to build in as glibc uses two parallel trees for source and build codes. This is nice so that if I have messed up my build I can just clear out that directory and still have the source code to start my bui...

Lab 2A: Building Open Source Software Packages

Image
     Today I will be building the open source GNU package less. Less is a really handy tool for file reading that lets you traverse backwards as well as forwards through the file. Unlike it's cousin more, less does not have to read the entirety of a file before starting so its great for quickly accessing large files.      The first step is to get the code for it on the server. I am building on AArchie so I will use wget to download the tar file (http://ftp.gnu.org/gnu/less/less-530.tar.gz):      Now I have the compressed tar file a simple tar -xzf less-530.tar.gz  command (-x for extract, -z for decompression with gzip, -f since we are decompressing a file) will unpack it so I can begin my build. I now have a less-530 directory in my working directory and once I move into it I can see the guts of the less package:      The safest choice from here to ensure I don't break my build is to check the README file to...

Lab 1: Investigating Open Source Development

     While most people today have at least heard of Open Source software, many people, myself included, don't know a lot about how the development process works or how these communities are structured and managed. Today's post will explore these topics in reference to two Open Source projects from different licences, Privacy Badger under the GNU General Public License and Deeplearning4j under the Apache2.0 License. Privacy Badger:      Privacy Badger is a browser extension aimed at striking a balance between a user's right to privacy and content providers' business interest. By only blocking ads and cookies that ignore the Do Not Track setting of the user's browser, thus as long as a content provider isn't hosting those types of ads they won't lose out on ad revenues.      Privacy Badger is run primarily off of its GitHub repository (link below) and it seeks to make contribution easy by keeping a list of desired issues to tackle that ar...