Raging Bits

Easy systems, raging bits

Migrate From Pelican to Jekyll

| Comments

After more than a year using Pelican I’ve just decided to move this blog into Jekyll. I’ll be using the convienient Octopress setup which nicely integrates with Github Pages.

Once everything in my server has been backup and had this running along for a bit with nothing going amiss, then I think I’d shutdown my paying server, and hopefully save some money (~100€ yearly).

Yeah, I know that I could have just started deploying my previous Pelican blog into Github. The main reason for a change is that Pelican required me a bit of fiddling and hacking around; which, ironically, is actually the reason I chose it in first instance. But this time, I want a little more features, for a little less effort. And if I end up forcing myself to dig into another system, I don’t want it to be in Python this time. Do not be mislead, I love Python; I just want to pop out of my comfort zone and see what happens.

The migration part

I had to convert about 3 years of writing (34 posts) stored as many text files, scattered within a 3 level hierarchy partitioned by year and month:

└── 11
    └── i-want-my-vim-back.md
└── 06
    └── exportacion-de-m3u-en-python-y-bash.md
└── 10
    ├── bjarne-stroustrup-c11.md
    └── nexus-one-miui-2.9.md

Each file having a header:

title: List directories within a dir in C
date: 2012-03-06 03:00
category: linux
tags: linux, c
lang: en

This setup is pretty close to the one that Jekyll uses, in which:

  1. Post file names must start with the date.
  2. The header have the same fields, but must be valid yaml at the start of the file, using --- lines as separator.

So it was just a matter of finding, copying and properly renaming all the posts from the Pelican source directory.

for post in $(find -name *.md)
  date=$(grep -i "date: " $post | sed -e "s/[dD]ate: \([0-9-]\+\).*/\1/g")

  if [[ ! -z $date ]]
      mv $post octopress/source/_posts/$date-$(basename $post)

The previous listing was prettyfied for displaying, the actual command was more like: for post in $(find -name *.md); do date=$(grep -i "date: " $post | sed -e "s/[dD]ate: \([0-9-]\+\).*/\1/g"); if [[ ! -z $date ]]; then mv $post $date-$(basename $post); fi done

And then

$ rename .md .markdown *.md

After that I had to manually add the --- separator to each post header. I couldn’t find a simple way of running a multiline sed replacement for all of them, so I opened all files in with vim *.markdown, recorded a macro, and runned it for all the 30 buffers. It took me about a minute or two.

Then copy the img directory, which contained the assets, maintaining the same relative path for all post images.

Using Jekyll/Octopress

Octopress is nothing but very convenient for slackers. You clone/fork their repo in github, and you already have a nice Jekyll setup. They have very detailed tutorials on their site, so the part of installing ruby, the gem and all that, I’m not repeating it here.

Once you’ve filled the ._config.yml file with your own data and have all posts within source/_posts, run:

  • rake generate will create static html with your blog.
  • rake preview will serve it on localhost:4000 for you to revise.
  • rake deploy will commit the generated html to the github repository you specified in the config.

¿My github repository? Yeah, create a username.github.io named public repository along with your own and put your domain within a CNAME file under source: echo "blog.ragingbit.com" >> source/CNAME if you’re using your own domain.

The html output will be commited to the master branch, the actual blog markdown code, will go to the source branch. So you rake deploy for publishing, and git push origin source for actually versioning your writings.

So, here we are, new hosting, new blog.

Install Jelly Bean MIUI 2.9.29 in Nexus One Using Blackrose

| Comments

I tried Ice Cream Sandwich on my Nexus One before summer, and despite its quirks and unstable behaviour from the early version I had (MIUI 2.5), I felt I couldn’t go back to Android 2.3 after getting used to it for a while.

When I tried to get a more stable ROM version (MIUI 2.9), I found lots of trouble, obscure fails aggravated by a considerable lack of documentation about how to go for it. So here is how I made it.


Android 4.1 Jelly Bean needs more space in the /system and /cache partitions than it’s originally allowed in the Nexus, so we’ll need a custom bootloader and resize those partitions before think about flashing the ROM itself.

Steps are:

  1. Download Blackrose
  2. Get fastboot to work
  3. Install Blackrose and resize partitions
  4. Install the ROM

Download HBOOT Blackrose

Download and unzip Blackrose and Blackrose Manual (check md5 checksums)

$ md5sum blackrose_120421.zip
f89de99e616a56e1bae29a9f8a190ee1  blackrose_120421.zip
$ unzip blackrose_120421.zip
$ ls blackrose_120421
binary  BlackRose  BlackRose.exe fastboot-linux-i386  fastboot-linux-i386.zip  other

$ md5sum oblackrose_manual_120421.zip
3c2bc61256aba2df1575ce2b6c66c977  blackrose_manual_120421.zip
$ unzip blackrose_manual_120421.zip
$ ls blackrose_manual_120421
fastboot-l  fastboot-m  fastboot-w.exe  go2.lol  go33.lol  go35.lol  hboot_220-16-200.nb0
hboot_blackrose.nb0  README.txt

Get fastboot to work

You’ll need fastboot working properly in order to be able to interact with your phone. The blackrose_manual version does include a fastboot implementation. We’ll use fastboot-l for Linux.

  1. Get your phone into fastboot mode. You should read fastboot on your phone’s screen. To enter into fastboot mode:
    1. Turn your phone off
    2. Turn your phone on while pressing the direction ball
    3. Enter into fastboot mode
  2. Connect your phone via usb to the computer. You should read fastboot USB on your phone’s screen.
  3. Check wether it’s detected by fastboot: $ ./fastboot-l devices HT066P800146fastboot If that does not work:
     1. Try with `sudo ./fastboot-l devices` (might be mounted with root permissions, you can [write a udev rule](http://wiki.cyanogenmod.com/wiki/Udev) to prevent this)
     2. Check your [fastboot setup](http://wiki.cyanogenmod.com/wiki/Fastboot), your [android sdk installation](http://wiki.cyanogenmod.com/wiki/ADB) and make sure that the  `android_sdk/tools` dir is within `$PATH`.

Install Custom Blackrose and resize

Once you get fastboot to work, run the Blackrose executable which comes with blackrose_120421, and install the Custom version, to resize the partitions on the same operation.

Mind the skip parameter, it’s important.

$ ./BlackRose skip
    * daemon not running. starting it now *
    * daemon started successfully *

    |   Nexus One BlackRose 120421  |
    |  Made by Lecahel(XDA-dla5244) |
    |    Dok-Do belongs to KOREA    |
    1    Apply stock/custom BlackRose
    2    Disable HBOOT flashing protect
    3    Uninstall BlackRose
    4    More information
    5    Exit
    Please make a decision:1

Choose 1: Apply stock/custom BlackRose

    Select which you want
    1    Stock BlackRose
    2    Custom BlackRose
    3    Back
    Please make a decision:2

Choose 2: Custom BlackRose.

error: device not found
Input BlackRose password(48 char max):

Ignore the error: device not found erro and insert a blank password. Just press Enter.

Nexus One BlackRose Editor
If you want resize partition, type [resize] and press enter key:resize
system size(MB):280
cache size(MB):20
label name(20 char max):ARL

Here is the root of all problems: size partitions.

  1. Set 280 for the /system partition. If you don’t do so, the installation will fail with symlink: some symlinks failed because MIUI 2.9 demands a lot of space.
  2. Set 20 MB for cache. Otherwise your phone might not boot and enter into a loop.
  3. Set whatever as the label.

    system size:280MB cache size:20MB userdata size:136MB label:ARL

    • Customize success sending ‘hboot’ (512 KB)… OKAY writing ‘hboot’… OKAY rebooting into bootloader… OKAY
    • Stock/Custom BlackRose has been successfully applied

Done here!

Install the ROM

  1. Get the MIUI 2.9.29 ROM (or the latest one from the MIUI website) $ md5sum MIUI.us_passion_v4.1_2.9.29_0xD34D.zip 6433fa6970e65e910c9ebd77cb81563b MIUI.us_passion_v4.1_2.9.29_0xD34D.zip
  2. Put the image into your sdcard root directory
     - You can do so from recovery itself by going to `storage` and setting the storage mode on.
  3. Now wipe both caches
  4. Use the install image from zip and select the ROM.
  5. Cross your fingers.
  6. Reboot your phone after the installation has finished.

Used links and further documentation:

Export Part of a Git Repo to a New One Keeping History

| Comments

I maintain my personal scripts (only a few by now) on a public repository on GitHub, so it forces myself to take them more seriously and keep it clean. It looked like a good idea back then, because all I’ve got were just not enough for a repo on its own. So I kept them together in separate directories under the same repo this way:

 |- README.md
 |- project1
 |   `- script1.py
 `- project2
     `- script2.py

And each commit in the repo clarifies which project modifies by preceding it with a code this way:

d65bf0e project1: Adds test_rotate function to test rotate verb
2943d69 project1: Removes unused TestFileOperations.newfile
dc37490 project2: Rewrites comments
9d00b0a project2: Fixes output-based tests redirecting stderr

But as some of them are now well grown up, like backup.py, I thought they deserved more visibility and wanted those specific directories out into a new repo… but keeping the history with them, and without the commit message code boilerplate.

There is one easy obvious way to to this, by (1) cloning the repository, then (2) using git filter-branch to remove the undesired directories and (3) to rewrite commits messages afterwards.

# 1: Clone repository
$ git clone --no-hardlinks /original-repo /new-repo

# 2: Filter away undesired stuff
$ git filter-branch --tree-filter "rm -rf project2" --prune-empty HEAD

# 3: Filter 'project1:' code from commits messages
$ git filter-branch --msg-filter "sed 's/project1: //' --prune-empty HEAD

The --prune-empty option makes git to discard commits that become empty after the filter and therefore are now useless.

You can se the difference from the original repo and the new one for backup.

SD-Card Emergency Encrypted Backup

| Comments

I must confess I am a little bit obsessive about data loss. I am not just obsessed, but paranoid. I like backups. I love them. I do it constantly, automatically, distributed and encrypted. I even write my own backup tools. It’s never enough.

I usually have a 500GB 2.5” hard disk connected to my laptop when I’m working at home, and use fcron and my backup.py to take a complete snapshot of the working environment every 5h or so. I also backup my backups on another hard disk, on another city, each time I pass by my family house. I use Dropbox too. I love Dropbox. But I do not trust it, I don’t rely on it. I want to keep my data myself, so I replicate the ‘Dropbox directory’ at my own home server, with regular backups of it, and at my family’s PC (with its own backup schema as well). Sensible data is encrypted by using ecryptfs because this way, data happens to be scattered across different places and although barely accessible, it wouldn’t take much to a geek with physical access to hardware to get it.

But as I said before, it is never enough. This way of making backups isn’t bad, It makes it difficult to completely lose everything. But what happens with last-time files and changes?. I often imagine an scenario (a bad nightmare, really) where I’m travelling, carrying all my stuff with me, and for some stupid reason, my bag falls to the road being swiftly smashed and crunched by an evil double-decker. There is not hard-disk anymore, either the laptop’s or the backup one. They’re magnetics flying fragments now. Dropbox happened to be unsynced since a day or so of no internet connection, my last backup at home is 3 days old, my sister accidentally removed some backups to make room for a movie and I was being quite productive these past blank hours at the airport. Data loss. Horreur. I wake up screaming, soaked in cold sweat.

I know it’s not very wise to move around with data and backups in the same bag, and actually I don’t. And I also know that the chance for absolutely everything to fail it’s quite low, but you got the point, and remember: I’m paranoid.

Then I was at a store and saw one of those 32Gb tiny flash pen-drives. 32Gb, that’s a lot. At least for my important data being about 7Gb. Then I thought I could regularly backup my physical disk backups, not only after some weeks, but every few days, into a pen I’ll always carry around inserted in my keyring, performing the copies manually.

But after the first moment, I didn’t like the idea. Besides of being a backup-freak, I am, as any other conceited programmer, quite lazy about repetitive tasks. Even further, I’m slack. If there is any task that I must perform completely unchanged more than 10 times, I’ll start growing annoyed about the 11th time. Actually I’m being self-indulgent here, let’s say 5 times. If I have to remember and manually perform each backup, I will screw it at some copy, forget about it, delay doing them or a combination of the previous. I wanted my flash copies, but I wanted them automated.

Next thing I saw was a 32Gb sd-card. They’re unlikely to die in an accident, they even survive being washed along clothes. They have this cell worn out issue, but hey, they’re also relatively cheap. It would be nice to have one of those inserted into my machine and perform copies to it. My laptop happens to have one of those integrated card-readers, so I just went straight to Amazon and ordered a class-10 32Gb for about £17.

Laptop integrated SD card-reader

This way I could have an absolutely up-to-date copy of all my data with me, automatically done at each system shutdown or every few hours. It won’t help me in case of losing my laptop, because it would be gone with it, but it can definitely save my butt if some drastic hardware failure happens. There are a couple of drawbacks for backing up to a SD card:

  1. Flash cells worn out on write. If you constantly write to a flash memory, you will exhaust it soon.
  2. It is exposed. The **** card suddenly pops out when careless manipulating the laptop. Or someone could take it. It may be lost somewhere, with all my data in it.

Fortunately, (1) incremental backups only writes few Mb each time, so it won’t be so bad for cells, and (2) encryption exists for a reason, I can always prevent all it’s contents from being taken.

Once I got my card, I moved it from vfat (it has issues with +4Gb files), to ext4 and created a ecryptfs private directory on it.

Once the sdcard has been inserted, it is auto mounted by gnome on /media/4785B20834, so let’s get the device number from the mountpoint.

$$ df -h /media/4785B20834 
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb1              30G   44M   28G   1% /media/4785B20834

Once we know it’s /dev/sdb1, (1) unmount it and (2) format it. I will format it using ext4 with no journaling. Journaling helps to keep data integrity, by keeping an index of last recently written files, but it means constantly write a list on the device. We don’t want that, remember?

$$ eject /media/4785B20834
# -O ^has_journal disables journaling
# -L sets the volume label
$$ sudo mkfs.ext4 -O ^has_journal /dev/sdb1 -L sdbk

mke2fs 1.41.12 (17-May-2010)
Filesystem label=sdbk
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1957888 inodes, 7817984 blocks
390899 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
239 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 

Writing inode tables: dataone                            
Writing superblocks and filesystem accounting information: 

$$ udisks --mount /dev/sdb1
Mounted /org/freedesktop/UDisks/devices/sdb1 at /media/sdbk

Well, we already have the card, let’s go for the encryption. ecryptfs allows you to encrypt a certain directory on the fly, storing the data encrypted, and mounting it on at some point on ‘clear’, so you can work with it normally, while still ciphering file contents underneath one by one and changing its names. ecryptfs can be found in the ecryptfs-utils package.

So let’s go for it, first of all (1) create the ecrypt directory by mounting it for the first time, (2) answer the questions that the program raises: I use the aes algorithm, with a 32 bytes key, no to plaintext passthrough and yes to filename encryption. You can see the settings for your new partition when it is created, and (3) take a look to your password’s signature (8e85340a483cb3ce) because you’ll recognize it in further steps.

    mkdir /media/sdbk/bk
    sudo mount -t ecryptfs /media/sdbk/bk /media/sdbk/bk

    Select cipher: 
->  1) aes: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded)
    2) blowfish: blocksize = 16; min keysize = 16; max keysize = 56 (not loaded)
    3) des3_ede: blocksize = 8; min keysize = 24; max keysize = 24 (not loaded)
    4) twofish: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded)
    5) cast6: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded)
    6) cast5: blocksize = 8; min keysize = 5; max keysize = 16 (not loaded)
    Selection [aes]: 1
    Select key bytes: 
    1) 16
->  2) 32
    3) 24
    Selection [16]: 2
    Enable plaintext passthrough (y/n) [n]: n    <- Nope
    Enable filename encryption (y/n) [n]: y      <- Yep
    Filename Encryption Key (FNEK) Signature [8e85340a483cb3ce]:  
    Attempting to mount with the following options:   ^-- Look!
    Mounted eCryptfs

Once you have it mounted, let’s put some sensible data in it, unmount it and see what happens.

# ex-girlfriend pic
cp ~/Photos/DSC0069.jpg /media/sdbk/bk
ls /media/sdbk/bk

# umount it
sudo umount /media/sdbk/bk
ls /media/sdbk/bk

Oh! so when we do not set the password and mount the directory, all we have in there are weird files. Just perfect. If you mount it again with mount -t ecryptfs DIR DIR, you’ll have to go through the whole set-up process again and answer algorithms, sizes and options to be able to mount. Second step is to make it mount automatically without having to type our password.

We’ll use the gnome-keyring for this. The gnome-keyring will pop out at system start-up asking for the keyring password, and once you have supplied it, applications can request passwords by its signature (ahh! 8e85340a483cb3ce) and use it without user interaction.

First, we’ll (1) add the partition to /etc/fstab, so it will mount automatically and configure it so the user can mount it without being root. Once we’ve got that, we (2) add our password to the gnome-keyring and (3) enjoy mounting/unmounting as we please.

# 1. Mount it and get the config from /etc/mtab
mount -t ecryptfs /media/sdbk/bk /media/sdbk/bk
Select cipher: 
1) aes: blocksize = 16; min keysize = 16; max keysize = 32 (not loaded)
2) blowfish: blocksize = 16; min keysize = 16; max keysize = 56 (not loaded)

(... yes, all this again ...)
Mounted eCryptfs
$$ ls /media/sdbk/bk
$$ grep ecryptfs /etc/mtab
/media/sdbk/bk /media/sdbk/bk ecryptfs rw,ecryptfs_sig=8e85340a483cb3ce,ecryptfs_cipher=aes,
        ecryptfs_key_bytes=32,ecryptfs_fnek_sig=8e85340a483cb3ce,ecryptfs_unlink_sigs 0 0
$$ sudo vim /etc/fstab

We add user to the options, so the user can mount/umount as he wishes. Leaving the file like this

# sdbk
/media/sdbk/bk /media/sdbk/bk ecryptfs user,noauto,rw,ecryptfs_sig=8e85340a483cb3ce,ecryptfs_cipher=aes,
        ecryptfs_key_bytes=32,ecryptfs_fnek_sig=8e85340a483cb3ce,ecryptfs_unlink_sigs 0 0

Well, we have it on the /etc/fstab, ready to mount at startup and at anytime we do mount -i. Lets add the key to the keyring to avoid typing it each time.

# 2. add key to the keyring
$$ ecryptfs-manager
eCryptfs key management menu
    1. Add passphrase key to keyring  <--- this one
    2. Add public key to keyring
    3. Generate new public/private keypair
    4. Exit

Make selection: 1

    Mount-wide passphrase:   <-- type your password
    Confirm passphrase:      <-- do it again
    Using the default salt value      
                                      v---- Look ma! same signature!
Added key to keyring with signature [8e85340a483cb3ce].

Now it’s time to test that everything works as expected. Lets (1) unmount it and then (2) mount it with no password annoyance, and finally (3) unmount it the same way.

# umount it as root
$$ sudo umount /media/sdbk/bk
# check whether we can mount it
$$ mount -i /media/sdbk/bk
$$ ls /media/sdbk/bk
$$ umount /media/sdbk/bk

Got it! This way is easy enough to get encrypted contents in the sdcard. All we have to do now it’s automate the backups via fcron. I use fcron and not just cron because its for a laptop. fcron works similar to anacron and remembers where it was at system shutdown. So if I set it to make a backup each 5h, work for 2 and then shut it down, next time it will keep the count and wait just 3h for the next backup.

Options frequency command
2 # if lavg5 <= 0.5 wait 1h max, start inmediately if it had to happen whilst off
3 @lavg5(0.5),until(1h),bootrun(true) 5h python2.6 backup.py --plan .hdbk.conf backup
4 @lavg5(0.5),until(1h),bootrun(true) 4h python2.6 backup.py --plan .sdbk.conf backup

I have it one each 5h and another each 4h. Starting only when system average load is less than 0.5. Backup!

EDIT: I’ve found that this configuration does not properly mount the ecryptfs file system on startup, and trying it with sudo mount -a it asks for the password. I’m obviously missing something. My solution was to add noauto to the fstab options so the partition won’t be mounted on startup and to add mount -i /media/sdbk/bk in my .profile file this way.

# Mount ecryptfs sdbk directory
if [ -d "$SDBK" ]; then
    mount -i "$SDBK"

Cambiar Email en Muchos Commits De Git

| Comments

Quería replicar un antiguo repositorio git en el cual había usado mi anterior email de cuando curraba en la universidad. Para ello debía reescribir todos los commits y cambiarles el email. Es bastante fácil, pero es mejor hacerlo sobre una copia del repositorio, y si metes la pata siempre puedes hacer un git reset --hard HEAD y listo.

Para el rewrite solo hay que usar git filter-branch, que permite este tipo de cosas peligrosas como son las reescrituras en masa.

git filter-branch --env-filter '
 GIT_COMMITER_EMAIL=<new.email@gmail.com>' --all

Digo que son peligrosas porque esto le pondrá tu email a absolutamente todos los commits, si tienes código de colaboradores, te cargarás sus emails. Si ese es tu caso, pon alguna condición en el filtro para evitar sobreescribir lo que no quieres.

Change Email in Multiple Git Commits

| Comments

I wanted to replicate an old git repo, in which I used as author my old university-work email, but using my current gmail account as email. Changing it for both commiter and author for all commits involves rewriting the whole repo. It’s not difficult at all, but better work on a copy of the repo, and remember that if you shoot yourself on the foot you can always get back by doing git reset --hard HEAD.

The rewrite can be easily done by using the filter-branch tool applied to all branches at once:

git filter-branch --env-filter '
 GIT_COMMITER_EMAIL=<new.email@gmail.com>' --all

This is the nuclear way, it will rewrite absolutely all commits. This won’t work straight if you have code from collaborators, then you’d lose their emails after the unconditional rewrite.

Activate Color in Git

| Comments

I’ve got sick of adding the --color flag to each git command I type so I went through the git config documentation and found several color.* options. Those options can be changed by using the git config VAR VALUE command, so initially I ran:

$ for CONF in branch grep diff interactive status;\ 
    do git config color.$CONF auto --global;\ 

To activate them all by setting each single color option to auto. But after reading a little more I found a much simpler way to have everything using color by default. Just simply type:

$ git config color.ui true

And that will do the trick. It is very nice and useful to read coloured output when using git, specially when reviewing diffs in the midst of a git add -p messy jam.

My .gitconfig ended up like this:

        ui = true

Recuperar Un Stash Borrado en Git.

| Comments

Si os pasa como a mi que confundís git stash pop (aplica y borrar un stash) con git stash drop (que borra directamente), no lloreis aún. git stash drop imprime el hash del commit que estaba en stash y que hemos borrado, y con el puede recuperarse.

git stash drop
Dropped refs/stash@{0} (e692be2bc305348380c7c71a80867539babad3d7)

Si tienes el hash aún, puedes hacer simplemente:

git stash apply e692be2bc305348380c7c71a80867539babad3d7

Y salvarlo.

La verdad es que he pasado mal un ratillo.

Recover a Lost Stashed Commit in Git

| Comments

If it happens that you confuse git stash pop (apply and remove an stashed commit) with git stash drop (simply remove it), do not cry yet. git stash drop yells the hash of the dropped commit, so you still have a chance to recover it.

git stash drop
Dropped refs/stash@{0} (e692be2bc305348380c7c71a80867539babad3d7)

If you manage to keep the hash, you can just do:

git stash apply e692be2bc305348380c7c71a80867539babad3d7

And save it.

So close.

List Directories Within a Dir in C

| Comments

When you just only want to list directories within a given directory in C, things can be not so obvious.

Using the non-standard function scandir its quite easy to get all files in a directory. The prototype looks like this:

int scandir(const char *dirp, struct dirent ***namelist,
    int (*filter)(const struct dirent *),
    int (*compar)(const struct dirent **, const struct dirent **));

Scary? Don’t worry, it just takes:

  • dirp: Path to the directory to list.
  • namelist: List of struct dirent * by reference. Uninitialized, scandir will alloc space for it. Should be freed afterwards.
  • filter: Filter function. Will be called per each dir entry which will be excluded from the list in case of filter returns 0. Takes a struct dirent * to decide.
  • compar: Comparison function. Used to sort entries. Takes two entries and returns < 0 , 0 or > 0. Lexicographical ordering its already implemented in alphasort, also included in <dirent.h>.

      #define _BSD_SOURCE 1;   /* Allows dirent.h scandir() */
      #include <dirent.h>
      int main(int argc, char * argv[]) {
          int i = 0;
          struct dirent ** filelist = NULL;
          int ndirs = scandir(argv[1], &filelist, NULL alphasort);
          if( ndirs < 0 )  /* Check errors */
              return 1;
          for(; i < ndirs; ++i)
              printf("Full path to file in %s: %s/%s", 
                  argv[1], argv[1], filelist[i]->d_name);
          if( filelist != NULL ) {
              for(i = 0; i < ndirs; ++i)
          return 0;

We’re ignoring the filter function and using the standar alphasort so it’s really easy to list all files within a dir (which includes special entries like . and ..).

The problem is that I just wanted to get the directories within the dir_path. To do this, I thought: “Ok, I’ll just implement a filter which calls stat and checks the file with the S_ISDIR(st_struct) macro”. But you just can’t. The filter function only takes a struct dirent *, which has an incomplete d_name with just the filename, not the complete path, needed for stat.

Not all is lost, thought. The struct dirent also has a d_type member which can be checked to be DT_DIR. The drawback for this is that it isn’t supported by all filesystems, so some of them may return a DT_UNKNOWN which would make me cry bitterly. The (incomplete) solution then, is to write a filter that checks d_type and then re-check the opening of these directories.

Will work like charm for most filesystems, won’t crash on the sloppy ones. I’ve been looking for the right way to do this in C, or even better POSIX C, but all I can find its the advice to write my own scandir, based on standard functions, which would be easy enough indeed, but not really the right way I’m always pursuing.

#define _BSD_SOURCE 1;   /* Allows dirent.h scandir() */
#include <dirent.h>

int filter(const struct dirent * dire){

    /* Discard . and .. */
    if( strncmp(dire->d_name, ".", 2) == 0
        || strncmp(dire->d_name, "..", 3) == 0 )
        return 0;

    /* Check whether it is a DIR or not.
    * Some FS doesn't handle d_type, so we check UNKNOWN as well */
    if( dire->d_type != DT_UNKNOWN
            && dire->d_type != DT_DIR )
        return 0;

    /* We've nothing against it. Accept */
    return 1;

int main(int argc, char * argv[]) {
    int i = 0;
    struct dirent ** filelist = NULL;
    int ndirs = scandir(argv[1], &filelist, filter, alphasort);

    if( ndirs < 0 )  /* Check errors */
        return 1;

    printf("Only directories\n");
    for(; i < ndirs; ++i) {
        printf("Full path to dir in %s: %s/%s\n", 
                argv[1], argv[1], filelist[i]->d_name);

    /* Check dir again using stat, opendir or... */
    if( filelist != NULL ) {
        for(i = 0; i < ndirs; ++i)

    return 0;

If someone has any ideas/suggestions/complains please, tell me. Please.