Angular/Node: Building a Command Line Tool to Generate Projects Part 2 - Angular and your FileSystem

Note: this is part 2 of a multi part blog about my quest to use angular for server-side programming, previous posts:

• Part 1 Angular / Node : Building a Command Line Tool to Generate Projects
  • Note: The code from part 1 will need a few updates to be compatible with the code from this post, but everything in its github repo is current, and should work along with the code below. So if you encounter issues with anything in this post, be sure you installed node-ng locally via npm (which will pull the latest code from GitHub) by running:

npm install node-ng

Time to Take a Look at What We Need from Node

One of the first tools we will build on our quest for a command line program to generate project skeletons for us will be an Angular module that wraps node's fs module. The fs module provides low-level access to the filesystem, so we will wrap it as an Angular module (which is what we will do anytime we run into outside code we need to integrate into our project).

Take note that from here on we will be using CoffeeScript.

Also, we must keep in mind that Angular was designed from the start as a frontend framework.

This helps explain (or justify, depending on how you may feel about it) many of the choices made by the team that created it then and now (since Angular is still being actively developed at this time), while also shedding light on why some features were kept while others were not.

This is why we have the angular.module.directive function, allowing us to manipulate the DOM, and associate it with our JavaScript, making all of our fun web page features Angular allows, ie: drag and drop, animations, etc. This is also why Angular's digest loop is focused around DOM changes, and does a lot of copying and replacing DOM nodes based on how you place your various directives in your HTML, so it's quite a powerful system.

Thankfully, there is more to Angular than just DOM nodes and the digest loop, otherwise, it might not be of much use outside of the browser. Angular has many features which will come in handy server side: DI, services, etc.

Now I will start building our command line tool, piece by piece, implementing each feature set as an angular module, installable via npm. Today we will work with node's file handling functions.

First, we need our app, and again please notice we are using CoffeeScript as it allows a cleaner more concise way to write JavaScript. If you are not familiar, I highly suggest taking a few
moments to read up on it.

Our FS Module Wrapper

The first thing we do is create our angular module object using the angular.module function.

node-fs.coffee

'use strict`

app = angular.module 'node.fs.app', []

Angular Promises in Node

Now we need a quick way to wrap the functions from node, into a form useable by Angular, as well as Angular's $q based promises. For the initial wrapper object, I will just inject the node fs module into an Angular service for us to use later. Then, for the promises, I will use the promisify function that I explained in part one
of this series. That function takes a single non-blocking function as its only argument and returns a new function that Angular can work with.

# first we need our service to call node functions from
app.service 'nodeFs',[->
    require 'fs'
]

# next we will create a factory that will wrap our functions
app.factory 'ngIfy',['nodeFs',(nodeFs)->
    (name)->
        promisify nodeFs[name]
]

# now we need to wrap a few of the basic fs functions for $q 
app.factory 'ngStat',['ngIfy',(ngIfy)->
    ngIfy 'stat'
]

app.factory 'ngReaddir',['ngIfy',(ngIfy)->
    ngIfy 'readdir'
]

# and we also need some private functions, that implement some of the 
# functionality we need, I am going to be showing that the 
# functions are private by starting their names with a single _
app.factory '_isDir',[->
    (res)->
        res.blksize == res.size
]
app.factory '_isFile',[->
    (res)->
        res.blksize !== res.size
]
# now we use those implementation functions to implement the 
# syncronys and asyncronys versions of isFile and isDir
app.factory 'isDir',['_isDir','ngStat',[(_isDir,ngStat)->
    (name)->
        ngStat(name).then (res)->
            actualResult = _isDir res
]
app.factory 'isDirSync',['_isDir','nodeFs',(_isDir,nodeFs)->
    (name)->
        _isDir nodeFs.statSync(name)
]
app.factory 'isFile',['_isFile','ngStat',(_isFile,ngStat)->
    (name)->
        ngStat(name).then (res)->
            actualResult = _isFile res
]
app.factory 'isFileSync',['_isFile','nodeFs',(_isFile,nodeFs)->
    (name)->
        _isFile nodeFs.statSync(name)
]

Reading and Writing to the Filesystem

Now that we are almost done, we only have a few more things we need here.

The next issue we will tackle will be reading from and writing to files,
for the moment we will avoid actually deleting files, because that's not
one of our requirements, but you can try that yourself as an exercise
after we're done. Also, we are not going to do blocking versions of these functions because reading and writing to files could really take a long time, so no need to block our progress while we do that.

app.factory 'readFile',['$q','nodeFs',($q,nodeFs)->
    def = $q.defer()
    (filename)->
        nodeFs.readFile filename,(err,res)->
            if err
                def.reject err
            else
                def.resolve res
            return
        def.promise
]
app.factory 'writeFile',['$q','nodeFs',($q,nodeFs)->
    def = $q.defer()
    (filename,data)->
        nodeFs.writeFile filename,data,(err,res)->
            if err
                def.reject err
            else
                def.resolve true
            return
        def.promise
]

Listing Files

And finally our last requirement from the fs node module is going to be listing directory contents. Which will be pretty easy as well. We just need to use the ngReaddir factory we defined earlier.

app.factory 'listDir',['ngReaddir',(ngReaddir)->
    (name)->
        ngReaddir name
]

Testing

Finally, we can write a few little tests to be sure things work as planned.

module.exports = ()->
    require '../dist/node-fs.js'

    fmtstr = require '../fstr.coffee'

    isDir = ng_load 'isDir',['node.fs.app']
    isDirSync = ng_load 'isDirSync'
    isFile = ng_load 'isFile'
    isFileSync = ng_load 'isFileSync'
    listDir = ng_load 'listDir'
    readFile = ng_load 'readFile'
    writeFile = ng_load 'writeFile'
    $q = ng_load '$q'


    # test data 
    firstTestItem = './dist/node-fs.js'
    secondTestItem = './dist'
    thirdTestItem = './src'
    fourthTestItem = './src/node-fs.coffee'

    testItems = []
    for i in [firstTestItem,secondTestItem,thirdTestItem,fourthTestItem]
        testItems.push i

    # blocking tests
    console.log 'Starting blocking tests\n'

    console.log '\ntesting isDirSync\n'

    testItem = (func,itm)->
        func itm

    testDirItem = (itm)->
        result = testItem isDirSync,itm
        "is #{itm} a directory? ---> [#{result}]"

    for itm in testItems
        console.log fmtstr(testDirItem(itm),"--",60)
    
    console.log "\ntesting isFileSync\n"

    testFileItem = (itm)->
        result = testItem isFileSync,itm
        "is #{itm} a file? ---> [#{result}]"
    
    for itm in testItems
        console.log fmtstr(testFileItem(itm),"--",60)

    doDirTests = ->
        testDirItem2 = (itm)->
            testItem(isDir,itm).then (res)->
                "\nWe are testing if #{itm} is a dir, is it? ---> [#{res}]\n"

        for itm in testItems
            testDirItem2(itm).then (res)->
                console.log fmtstr(res, '--',80)
            ,(err)->
                console.log "ERROR@!:",err

    doFileTests = ->
        testFileItem2 = (itm)->
            testItem(isFile,itm).then (res)->
                "\nWe are testing if #{itm} is a file, is it? ---> [#{res}]\n"

        for itm in testItems
            testFileItem2(itm).then (res)->
                console.log fmtstr("#{res}", '--',80)
            ,(err)->
                console.log "ERROR@!:",err
            console.log "counting #{testItems.indexOf(itm)}"

    doListDirTests = ->
        listDir('./dist').then (res)->
            count = 0
            count++ for r in res
            console.log "Should have 1 file, had: #{count}"

        listDir('./src').then (res)->
            count = 0
            count++ for r in res
            console.log "Should have 1 file, had: #{count}"

    doReadWriteTests = ->
        tstTxt = "this is a test file"
        tstName = "tst.txt"

        writeFile(tstName,tstTxt).then (res)->
            readFile(tstName).then (res)->
                console.log "Data from #{tstName}\n#{res}\n\nData from variable\n#{tstTxt}"

    console.log '\nstarting non-blocking tests\n'
    do doDirTests
    do doFileTests
    do doListDirTests
    do doReadWriteTests

fmtstr function

You may have noticed that I'm using a function fmtstr, which I am
pulling from fstr.coffee. This is just a string formatting abstraction
which I felt was so unrelated to this project's goals that it needed to be in its own file. At any rate, here are the contents of fstr.coffee:

makePadd = (char,num)->
    res = []
    for x in [1..num]
        res.push char
    res.join ""

fmtstr = (s,splitchar,size,padchar)->
    splitchar = splitchar or '--'
    size = size or 40
    padchar = padchar or ' '
    #console.log s
    parts = String(s).split(splitchar)
    padlen = size - (parts[0].length + parts[1].length)
    padd = makePadd padchar,padlen
    "#{parts[0]}#{padd}#{parts[1]}"

module.exports = fmtstr

Final Notes

We have all we need from nodes fs module for now! Let's take a look at the different things we've managed to accomplish in this article:

• checking if the file is dir or not

• reading files

• writing files

• and listing directories

• and all of these features are integrated into the Angular workflow

Thanks for coming along on this journey with me, we have made progress, but still have a long way to go.

This was part 2 of an ongoing saga, where I attempt to use Angular serverside.

Join me next time, when we will be working on rendering templates from strings and files, in part by using the module we coded today, node.fs.app

Again, all of the code from this blog post can be found on GitHub => here