Category: Web Design
I’ve been getting quite a few questions about jQuery.continuations after my last round of posts and the lightning round I gave at ADNUG last month. I have this horrible habit of pretending that I’m a lot more articulate than I really am. Obviously my last few posts didn’t do this library any justice.
I think Jimmy put it the best when he said: “Yeah, I read your posts but there just wasn’t a whole lot of code”. So this time I’m going to write a post that’s mostly code and see if it helps.
Show me something concrete
Let’s start with an example and then I’ll explain what’s going on:
If you run this example, you’ll be shown a dialog. Clicking “Success” or “Failure” will demonstrate the behavior of a successful or failed http post (respectively).
NOTE:
I’ve noted this in the code several times but here it is again: don’t bother with the “Server” stuff. It’s just a helper to mock the ajax calls since we’re not using a server here.
Custom options
We are making use of the correlatedSubmit method here (an extension to jquery.forms). Notice that we’re passing in the custom “closeDialog” property.
Passing in a hash to correlatedSubmit or as the “options” property of the hash to $.ajax provides a local set of custom options that are made available during the processing of the clientside continuation.
Great, what does that mean? Well, here’s how this works:
Whenever a continuation is received (via the success callback from $.ajax), this custom dialog policy will be inspected. Notice that we’re matching against “continuation.options.closeDialog”. The options property of the continuation contains the same options that you passed in so in this case, “closeDialog” would be true.
Note:
My first pass at this didn’t have these custom options and our conventions were ALWAYS global. You should ask Jeremy what he thought about that.
In case you missed the point here, our policy conventionally closes a jquery ui dialog when the response was successful.
Reusable conventions
Conventions are only “conventions” when they’re reusable. Reusable pieces of code really strut their value when they’re reusable in more than one project.
The “Failure” button demonstrates a failed continuation response from the server. fubuvalidation-js has a built-in convention for detecting those errors and rendering them into the form that originated the request. So simply including fubuvalidation-js into this jsfiddle was enough to get that to work.
From client to server
The validation and “close dialog” policies are pretty useful. For us, they’re very useful because we use them throughout our app. These are just two simple examples of what can be done here. Let’s brainstorm a few more to make sure I’m articulating the power of this correctly:
- Refresh the page (force a full refresh on certain conditions)
- Redirect to a new url
- Display errors (“something went terribly wrong”)
- Keep track of pending ajax requests
The list goes on and on but let’s talk about this “redirect to a new url” idea.
APIs and Continuations
A common technique found in HTTP-based APIs is to provide a set of information regarding a resource when it is created. For example, “POST /users/create” may return things like:
- The ID of the User
- Some urls (e.g., “go here to view the details”, “go here to edit the user”
If you design your endpoints to follow a similar technique, you can plugin policies that use the response and orchestrate the behavior of your system from a central block of code. Of course, in small applications this doesn’t pay off but in large and complex systems the value starts to shine.
Wrapping it up
Hopefully the code speaks for itself and this sparks a few ideas.
We have finished the integration of Hyper-V Bandwidth Meter and Hosting Controller (web hosting control panel).
- Bandwidth reports (created by Hyper-V Bandwidth Meter) are now provided to view with Hosting Controller.
- In case of bandwidth over usage, Hosting Controller will take appropriate action (configurable) e.g. Suspend Virtual Machine, Resume Virtual Machine.
As JavaScript applications become larger and larger, we have to think about the download size, memory usage and other performance constraints for our end users. There are a number of aspects to consider, one of which is how to deliver the HTML templates that your application is using, to the user’s browser, so that your application can render the templates.
For my smaller applications, I tend to stuff a number of <script> blocks in to the HTML that the user downloads. This makes it easy for me to work with and I don’t have to worry about whether or not the template exists. But when the number of templates gets to be more than 5 or 6 small ones, this gets out of hand quickly. It makes it hard for me to manage them as I have to scroll through a lot of template code. Putting them in external files and then including them in the page with some server side technology helps the developer problem but it doesn’t solve the client problem of having to download all of these templates even if they are never used.
To deal with the issues surrounding templates in HTML files, we can split our templates in to separate files and then use asynchronous calls to our server to load them as needed.
Backbone.View Render Semantics
One of my goals, other than the asynchronous template loading, is to keep the general semantics of a Backbone View’s `render` method. It’s a short list, but it’s an important list as most of the Backbone community expects the render method to work this way.
Semantics are generally important as they give us information about how methods and objects are expected to be used, as well. This, in turn, informs the method signature and behavior. And all of this comes back to the Liskov Substitution Principle (LSP) from the SOLID principles, which tells us that we need to pay attention to semantics so that we can drop in replacements as needed.
The general semantics and method signature of a view’s render include:
- No parameters: the render method shouldn’t require any parameters. You should be able to call `view.render()` and have it work fine
- Chainable: the render method should return `this` so that it can be chained with other method and attribute calls. This is most commonly done as `view.render().el` to grab the element that was rendered to
- Populate `el`, don’t replace it: the render method should populate a view’s `el` with any contents that the view needs to display. It generally shouldn’t replace the `el` as a whole
Aside from these three items, there’s a lot of flexibility in how a view will typically be rendered. There’s also plenty of room for interpretation and divergence from this list. Many applications use a render method signature that takes parameters, or that replaces the `el` entirely. When changes like this are made, it’s a good idea to document them so that people will know why the changes are in place.
The benefit of keeping these semantics, though, is that you can swap out a synchronous, pre-loaded template rendering view with a view that uses an asynchronous template loading mechanism. Or, better yet, you can have an intelligent system that uses asynchronous calls to get the template the first time it needs it, and then uses caching to keep the template around and do synchronous rendering on subsequent requests for this view / template. If the semantics for the view are kept in place, it doesn’t matter how the view implements the rendering. The view can be dropped in or removed as needed, without having to change the surrounding code that uses it.
Simple Async Template Retrieval
We can keep this very simple, to begin with, using jQuery’s AJAX calls to load the template with a callback to do the actual rendering after the template is loaded.
Backbone.View.extend({
template: 'my-view-template',
render: function(){
var that = this;
$.get("/templates/" + this.template + ".html", function(template){
var html = $(template).tmpl();
that.$el.html(html);
});
return this;
}
});
In this example, we’re assuming that the view has a `template` attribute. This attribute represents the file that will be loaded from the server, and that file contain the actual template to be used.
We’re also using a convention of `/templates/{name}.html` for the template location on the server. This can be implemented easily in many different web server technologies. In Sinatra, for example, you can create a “public/templates” folder and put your HTML template files directly in that folder. They will be available without having to do anything more than start the Sinatra server.
When the call to `render` is made, the code makes an AJAX call back to the server to retrieve the specified template. A callback method is provided – and at this point, it assumes a successful call to get the template. When the template is returned from the server, the callback is fired, and the standard render code (using jQuery templates in this example) is executed.
Note that the `el` for the view is populated inside of the callback, but we are still calling `return this` at the end of the function. Even if we chain access to the `el` from the `render` method and immediately add the el’s contents to the DOM, this will still work:
$(“#content”).html(view.render().el);
The reason this works is that we are only populating the `el` with contents. We are not replacing it. When the AJAX call finally returns with the template, rendering it and populating the `el` will show the contents immediately because we’re setting the `html` method of an HTML element that is already attached to the DOM. It’s as if we had called `$(“#content”).html(“<div>some html</div>”);` directly.
Caching Templates
Now that we have a template loading asynchronously, it would be nice to only load it once instead of every time it needs to be used. This will improve the overall performance of the application, from the user’s perspective.
To do this, we’ll need a little more than just some code in the render method. We want to re-use templates that we’ve already loaded, which means we can’t just store the template on the view instance. We need to store it in a place where any view instance can grab a copy of the template if it exists, or have an asynchronous call back to the server done to get the template when needed.
TemplateManager = {
templates: {},
get: function(id, callback){
var template = this.templates[id];
if (template) {
callback(template);
} else {
var that = this;
$.get("/templates/" + id + ".html", function(template)){
var $tmpl = $(template);
that.templates[id] = $tmpl;
callback($tmpl);
}
}
}
}
The template manager object has one primary method that we call: `get`. This method takes in a template name to be loaded and a callback method that is executed when the template is found. By using a callback method instead of returning a value directly, we can ensure both synchronous and asynchronous calls will work correctly.
When you call `get`, it will check a hash / object literal to see if the template you want is already loaded using the template name that you provide to make this check. If it exists, it executes the callback immediately and passes the template along. If it does not exist yet, an AJAX call is made with jQuery to get the template from the server. Once the template is loaded, the callback that you passed in will be executed and the template is passed to it.
We can now update our view to use the template manager:
Backbone.View.extend({
template: 'my-view-template',
render: function(){
var that = this;
TemplateManager.get(this.template, function(template){
var html = $(template).tmpl();
that.$el.html(html);
});
return this;
}
});
This isn’t much of a change from the view’s perspective. It’s better encapsulated, though, and a little easier to read. The real work is now being done in the TemplateManager object and we can change how it behaves as needed without having to update our views.
Beyond Simple Caching
There’s one remaining problem that this code has. If you have multiple instances of a view requesting the same template at roughly the same time, multiple AJAX calls will be made – one for each instance of the view. The net result is a slowdown in the application’s performance from too many network calls, and also a visual oddity where the views will appear one at a time as the AJAX calls finish. This can be a pretty big drag on performance and UI responsiveness.
Thanks to some previous digging in to jQuery’s deferred and some help from Steve Flitcroft (@red_square), I was able to solve this problem fairly easily. I put the following code in to my BBCloneMail app, which uses my Backbone.Marionette plugin. It’s not directly implemented in Marionette’s `TemplateManager` but most of what you need is already there. There’s only a few additional lines of code that you need to make this work.
(function(){
var promises = {};
// Cache the returned deferred/promise by the id of the template
// so that we can prevent multiple requests for the same template
// from making the ajax call.
//
// This code is only safe to run synchronously. There exists a
// race condition in this function, when run asynchronously,
// which would nullify the benefit under certain circumstances.
var loadTemplateAsync = function(tmpId){
var promise = promises[tmpId] || $.get("/templates/" + tmpId + ".html");
promises[tmpId] = promise;
return promise;
}
// Use jQuery to asynchronously load the template.
Backbone.Marionette.TemplateManager.loadTemplate = function(templateId, callback){
var tmpId = templateId.replace("#", "");
var promise = loadTemplateAsync(tmpId);
promise.done(function(template){
callback.call(this, $(template));
});
}
})();
The basic idea is that I’m using a jQuery deferred / promise to fire the callback method from the loadTemplate parameters. To prevent multiple requests for the same template heading back to the server, I’m caching the promises by the template id. When a call to loadTemplate is made through the template manager, I check to see if I have a promise for that template already. If I do, I register the loadTemplate’s callback parameter with the promise. If I don’t, I create the promise and then store it by the template’s Id. Either way, I register the callback with the promise which guarantees it will be executed. Once the template is returned from the server and the promise is fulfilled (resolved), all of the callbacks are fired off with the template data and everything renders correctly.
Note, though, that this code is not 100% safe. If you call the template manager from code that is already asynchronous, you can end up with a race condition where multiple promises are created and multiple calls to get the template are done. You’ll still get all of your views rendered just fine, but this will eliminate the benefit of using a promise to reduce network calls. Calling this code synchronously, though, won’t cause this race condition and the templates will only load once before they are cached and re-used.
See It In Action
If you’d like to see an example of this code in action (including the deferred/promise code from above), check out my BBCloneMail application and it’s source code. You’ll see the update to the TemplateManager’s `loadTemplate` function in the code. When you view the live site, switch to the Contacts view and refresh the page. There will be a small delay in the template loading, and then all of the contacts (which all individually requested the same contact template) will all display at once. Subsequent requests to those areas of the app will be cached, of course, improving the user experience and performance even more.
In my effort to make myself an ex-GoDaddy customer, I have 3 domains left to transfer or let expire.
Two of them I’m making available to transfer to anyone who wants them (rather than letting them expire so that you would have to pay the recovery fee).
Anyone want these domain names before I let them expire?
whackcoder.net
startingwith.net
Both are currently parked and have no viewership to speak of.
After two posts to a reaction of a comment on feedback I had for the Git installer, I thought it was pertinent to share my thoughts on OSS rules of engagement. Scott Chacon left a comment along the lines of “your time spent complaining is better directed at submitting a pull request”. Which, as an OSS developer and project maintainer, I agree up to a point, until that reaction discourages feedback.
I understand how daunting it can be to contribute to an unfamiliar project, especially if you recognize that the people most familiar with the codebase would take a lot less time to do so, and probably do it “right” in the first place. I also understand how frustrating it can be to have large, visible OSS project where the number of people complaining/giving feedback far dwarfs the number of people actually contributing.
So what are the rules of OSS engagement for me?
1. Un-actionable complaints will largely be ignored
Mostly because I don’t have too much time to figure out what you want. I generally will respond with prodding, but unless you respond back, I won’t have anything actionable to address. Having an example of the alternative is generally the minimal bar here.
2. Feedback is encouraged. Example client code even better
As an OSS developer maintaining libraries, I often get feedback of “Can it do XYZ?”. Sometimes the answer is No, but I would like to see what your vision of how it should behave.
What should the methods be called? What does the configuration look like? What’s an example of the behavior you’re envisioning?
I don’t even care if the code compiles – just show me an example of what you’re thinking.
3. Unit tests for bugs encouraged, but not required
I just really need code to reproduce the bug. A report of a bug with zero code is much harder to reproduce, and much less likely to get fixed.
If you want to up your odds of getting a bug fixed, the more code I get demonstrating the problem, the better.
4. The more actionable the feedback, the more likely it gets integrated.
Pull requests are at the top end of the “actionable feedback” scale. An email reporting a bug with no code as at the low end. The more actionable the feedback I receive, the more likely I will have the time/energy to respond to the feedback.
However, I won’t turn people away for not giving me a pull request. I encourage it, and often point folks to try, especially if they want to “up the odds”.
I might encourage you to send me a pull request, but I’ll never disparage you for not sending me one.
One of my favorite shows on TV these days is (don’t laugh) the show Property Virgins on HGTV. In it, an experienced Realtor walks first-time home buyers through the house selection and offer process.
A lot of times the “let’s watch a couple pick a house” type shows highlight the inexperience of the buyers. Buyers tend to focus on the wrong thing, like paint colors or light fixtures, and gloss over things that are hard to change, like room layout. But the best part of the show happens right at the beginning, in a brilliant move to reset the usually unrealistic buyer expectations.
At the beginning of the show, the host asks the buyers a few key questions to identify both their budgetary constraints as well as their aspects of a home they most value. It could be location, layout, lot size, etc. But the buyers always have some sort of dollar amount they won’t go over.
The next step the Realtor does is rather brilliant – they take the first-time home buyers to a house that meets all of their aspects for an ideal home. It has the perfect layout, the perfect location, all the amenities they want, in the right condition, at the right size and so on. The buyers inevitably love this home, at which point the Realtor asks the buyers to guess how much the home costs. Also inevitably, the buyers are pretty far off the actual price, and inevitably their ideal home is typically at least double, if not triple or quadruple the buyers’ budget.
And inevitably, the buyers have sticker shock!
Talking buyers off the ledge
This fantastic approach accomplishes two important goals:
- Convey what perfection costs
- Force a prioritization
The host is never mean about showing the expensive house, but instead presents it as something to aspire towards. This also resets in the buyer’s minds that every house they see from there on out is going to have some subset of their valued aspects.
But instead of discouraging the first time home buyers, this approach tends to force them to focus on what is most important to them in a house.
Resetting expectations
We often have clients that are first-time software buyers, or first-time-with-someone-who-knows-what-they-are-actually-doing software buyers. A lot of what we do in the initial part of the project is framing the project for success. We look at everything the client wants, talk about scope and budget, then reset expectations back down to an attainable level.
What we don’t want to see happen are those buyers that look at dozens or even hundreds of homes looking for that house that checks every single checkbox and comes in at their budget. That house might exist, or it might not, but a lot of time can be wasted searching and searching.
Constraints force prioritization and hard decisions. Having an experienced guide (like that Realtor host) ensures that the buyer understands what’s feasible for their budget, as well as the guiding hand on helping to share experience on what things matter (the electric wiring needs replacing) or do not (the bedroom has shag carpet).
Delivering value is really only half the equation. It’s up to us as developers to make sure the buyer understands what they are getting for their money (or time), relative to what’s out there in the market. If you bought a house in isolation, it’s hard to know whether you’re getting taken to the cleaners. But by having frames of reference (Product XYZ is similar to yours, and took ABC man hours/dollars to build), we can center the conversation around “what’s most valuable to me” instead of “how can we squeeze everything in”.
Most Backbone application examples that you find on the web are very small and very simple. They tend to use a very simple idea of an application “init” method to start up a router and have the router kick off the real application code. I find this path to be very constraining, though. For the most basic of applications it works perfectly fine, but when you get in to anything of any real size and complexity, it becomes difficult to work with.
My Backbone.Marionette add-on helps to alleviate this problem by introducing the idea of application initializer callbacks. I’ve talked about these before, and I use them quite extensively in my applications. It really helps to keep my code clean and separated. And I recently realized that there’s an unintended benefit to using initializers like this: explicitly identifying the different stages of starting an application.
The 3 Stages
The three stages that I have identified in my own applications are (in order):
- The Application’s Foundation
- The Application’s Initialization
- A Contextual Start (optional)
Honestly, I could break this down in to a significantly larger number of stages if I really wanted to get fine-grained and detailed. More specifically, it’s the 2nd stage that can be broken down in to pre-initialization, initialization, and post-initialization. But I think as a general place to start, identifying 3 primary stages for an application’s startup is a good idea. It gives us a better understanding of what code should go where, helps us keep our code clean, improves re-usability and flexibility, and will likely help us scale our apps as well.
Stage 1: The Application’s Foundation
Consider this to be the stage where the browser is simply grabbing all of the specified resources from the server and running through the first pass of parsing the HTML, laying out what the static HTML has provided, styling it with the basic CSS definitions, displaying any images and text from the static HTML, and parsing the JavaScript files so it can get ready to run them. I’ve never seen an HTML application that didn’t have this stage in it, because without it, your browser doesn’t have anything to do or show.
This is the one step the everyone does, and does well.
Stage 2: The Application’s Initialization
Every application has an initialization stage. Some are more elaborate than others. At times you can get away with the over-simplified “app” object with an “init” method on it, like many of the sample apps and boilerplate codes will suggest:
App = {
init: function(){
// app initialization and startup goes here
}
}
App.init();
But when you get in to an application of any significant size and complexity, the single “init” method becomes a junk drawer where you tend to throw code because you haven’t yet identified where it should belong. Using a series of small initializer functions for each area of your application will help to alleviate this problem, as I’ve mentioned before. But the real question is:
What goes in the initializers?
Initializers are the bits of code, functionality, data and display that are absolutely required for your app to do it’s job, no matter what part of the application the user is trying to load up and use. They are the bits that must be initialized before the user can do anything meaningful with your application.
If your app has a menu structure generated by Backbone code, and it must always be present in the application, this should be in an initializer. If you’re building a multi-room chat application and you need to list the rooms that the user has favorited in a small block on the screen that is always visible to the user, this should be an initializer. If you’re building an image gallery and you need to load a thumb-nail list of images to show, no matter which image the user is trying to view, this is an initializer.
Other parts of your application code that the user doesn’t directly see may also be contenders for initializers, too. For example, if you have a router that needs to be up and running, the router probably needs to be instantiated inside of an initializer. But – and this is an important but – the routes on that router should not be executed during initialization. Instantiate the router and wait until after initialization has completed to call the “Backbone.history.start()” method, kicking off your route handlers.
Stage 3: A Contextual Start (optional)
I call this stage optional because not every application has a contextual kick off. Sometimes an application only needs initializers. The classic “todo” application is a great example of this. There is no contextual start up for this app. When the application is loaded, it initializes itself with the list of to do items and then it waits for the user to interact with the list.
When a Backbone application uses a router to respond to url pushState or hash fragment changes, though, it does have a contextual starting point. Each route that a user is allowed to bookmark or copy as a direct link will provide the context that our application needs to use, to get the user back to where they want to be. Even if a Backbone app has a router and contextual start, though, it may not be used. If there user hits the root of the application, there may not be any additional code to run for the empty (“”) route. When the user hits a route, though, that route must server up the context and application state that the user expects to see.
The problem that I see in most routed Backbone applications, though, is that they bundle the application initialization with the contextual start. That is, people tend to use the router and it’s callback methods as the sole place to get the application initialized and get the user back to the context of the route that they requested. For trivial applications, this may be fine. The initialization code may be so small that it doesn’t really matter if it’s crammed in to the router. But for any real application with any amount of complexity, this is a bad idea. It couples two very distinct parts of the application startup very tightly, and it can lead to bloated and unmaintainable routers with limited entry points in to the application.
So… what goes in the contextual start, other than just saying route callbacks?
Your route callbacks should be as simple as possible. They shouldn’t initialize your system as a whole. Rather, they should be used to determine the state of the application that the user wishes to see. Therefore, the code that goes in to a route callback should be the smallest amount of code that you can write, to get your application from it’s initial state (the state that it was in when the initializers completed) to the desired state.
In an image gallery application, a user may hit a bookmark that points to a specific image. For example, “#images/4″. The route callback that executes should load the requested image and display it on the screen:
Backbone.Router.extend({
routes: {
"image/:id": "imageById"
},
imageById: function(id){
var image = imageCollection.get(id);
App.showImage(image);
}
});
In a multi-room chat application, a user may hit a bookmark that should take them directly in to a specific chat room. For example, “#backbone”. The route callback that executes should take this room name and call the code that is necessary to enter the chat room.
Chances are this is a fairly involved set of code. You’ll need to load the list of users in the chat room. You’ll need to clear the current chat windows and possibly pre-load recent messages to be displayed. You may also need to change a browser’s websocket event listeners to pick up events for this specific room instead, so that messages for this room can be displayed as they come in.
This is a lot of code to run, and it’s fare more code than should be allowed in a route callback method. It should, then, be encapsulated in an object that is responsible for registering the user as having entered that chat room. This object is then called from the router:
Backbone.Router.extend({
routes: {
":roomname": "chatroom"
},
chatroom: function(roomname){
ChatApp.enterRoom(roomname);
}
});
Furthermore, there’s a high likelihood that the user will be able to enter chat rooms using some interaction on the web page. They may click on a chat room name in their favorite’s list, or they may type in a command like “/join #backbone” in to the chat application’s command area. For any of the the multiple ways to enter a chatroom, the code that is executed should be the same. Having the route callback be as simple and stupid as possible will promote code re-use and allow these three options to be easily implemented. If you only need to call “ChatApp.enterRoom(‘someRoom’)” to enter any room that the user specifies, providing options for how the user enters a room becomes trivial.
Putting It All Together
There are many different ways of stringing these different stages together. There are many different tools that help us to easily create the HTML that we need, include the CSS and JavaScript assets, and kick things off. Some applications are small enough that an initialization step and contextual start step can be combined. Most applications of any significant size, though, should think about separating these in to explicit steps in order to promote better architecture, clean code, and re-use of code.
For me, I’m accomplishing stages 2 and 3 with my Backbone.Marionette add-on. It provides a very clean way to create initializers:
App = new Backbone.Marionette.Application();
App.addInitializer(function(){
// add some app initialization code, here
});
App.addInitializer(function(){
// more initialization stuff
// for a different part of the app
});
// run all the initializers and start the app
App.start();
At the time of this writing, though, I don’t have an explicit “startup” callback mechanism in place. I have a “start” method on my Marionette.Application object. But right now this really only fires up the initializer callbacks. So, I’m currently using the “initialize:after” event to run my start code. I see this as a problem, though – not of a technical nature, but of a semantic nature. And yes, semantics are important.
App = new Backbone.Marionette.Application();
/* ... initializers go here ... */
// contextual startup
App.on("initialize:after", function(){
Backbone.history.start();
});
I’m not sure what the new implementation will look like, but it may end up being a “startup” callback method, similar to the initializer methods. The startup callback (or callbacks?) would execute after all of the initialization is done. This will provide a cleaner separation between the two concerns of initialization vs startup. I’ll have to see where this leads me, though, as right now, the majority of what I do for a contextual start is simply call “Backbone.history.start()”, as I showed in the above example.
Rails wants you to put specific files in specific folder structures, based on the object type that will be in the file. Java demands that files in a folder structure are namespaced by that folder structure. VisualStudio also makes it seem like file / folder names should be the namespace / class name as well – but that’s just a good suggestion and not a requirement. Other languages and frameworks have some requirements for how you organize your files, too… with the exception of browser based JavaScript (… mostly…)
JavaScript File Organization: You’re Doing It Right
The reality of JavaScript in general web-based development (not talking about server side at all, here) is that it makes absolutely no difference how your files and folders are organized. Zero difference at all… at least as far as the JavaScript runtime is concerned.
The only thing that matters is that you include your files in your HTML with a <script> block, correctly. You also need to pay attention to which order they are included in most cases, to make sure things are defined before they are used (tools like RequireJS and other script loaders and module definition / loaders help with this).
What does this really mean? It means:
You’re Doing It RIGHT!
Yup. You’re doing it right, because it doesn’t matter how you do it.
What does matter, though, is that you and your team (if you have a team) pick an organizational convention and stick with it. It’s actually very more important for your team to have a good file and folder structure for your JavaScript. But you don’t need to worry about what that structure is. Pick a standard and use it. When it fails (and it will), re-work your structure so that it works within the newly understood constrains and move on.
Of course, that fact that your browser and it’s JavaScript runtime don’t care about your file and folder organization doesn’t I’m without my opinions on how files and folders should be organized.
For example…
M, V and C Folders
A lot of people organize JavaScript MV* application files in a folder structure like this (BackboneJS in this case):
To the best of my knowledge, this folder structure is based on the “models”, “views” and “controllers” folder structure that was popularized by Ruby on Rails. Sure others may have had it first, but it was Rails that made it popular. Other MVC framework followed suit and demanded that you put your controller objects in the controllers folder, your model objects in your models folder, etc. But unless you’re Rails (or another framework that wants to be like Rails), this folder structure is stupid.
I’m pretty sure that Rails uses this folder structure to assume the types of objects that are found within the files. And I know for sure that it uses file names to assume the class that will be defined within the file. That is, when rails sees a file called “/app/controllers/foo_controller.rb”, it expects to find a class called “FooController” and it expects that this class will inherit from some Rails controller base class. If these expectations are not met, errors are thrown to say so.
I understand why Rails does it this way: file and folder based conventions make it easy to assume what a file will contain, and that makes it easy for the runtime to optimize for performance when pre-loading and caching the code contained within the files. This makes sense to me in Rails because the convention is based on good ideas for optimizing the way Rails works and the way it looks for files and how it loads them.
But, unless you’re Rails or another framework that wants to assume certain files in certain folder contain certain code, this is a terrible way to organize files.
The Junk Drawer
There are some good examples of other standards along this line. For example, I tend to follow the convention of a “public” folder with “css”, “images”, and “javascripts” folders. But honestly, this folder structure exhibits many of the same problems of being stupid that organizing files in M, V, and C folders does.
The real problem with these types of folder structures is that they become junk drawers. Even DHH and the Rails core team recognize that this is a poor folder structure outside the confines of Rails+Ruby code. That’s one of the reasons they added the Asset Pipeline in Rails 3.1. DHH even called the “javascripts” folder a junk drawer, very directly, in a RailsConf keynote in 2011 (or was it 2010?) – complete with a slide showing a drawer full of junk.
With any application that moves beyond a trivial number of files, these content-type, mime-type, code-type and general type-based folder structures turn in to a bloated pile of junk that is very difficult to sift through. Who wants to look at a folder with 20, 50 or 100 files in it, when you only really care about 2, 5 or 10 of those files?
And what happens when you suddenly have an object type that doesn’t fit your pre-established conventions? You end with a “lib” folder, like Rails, which becomes the ultimate junk drawer. “It’s not a model? It’s not a controller? It goes in lib.” – no matter what the actual functionality contained within the file is. The “lib” folder is asking to be a junk drawer… demanding it, really. So, do you follow that same junk drawer convention for non-M, V or C type-based files in your JavaScript apps? That doesn’t any make sense to me.
How I Organize Files: By Functionality
I prefer to organize my JavaScript files the way I used to organize my C# files in .NET projects: by functional area of the application. That is, I group files together in folders based on the area of the application that they facilitate.
For example, my BBCloneMail application has the following folder structure for it’s JavaScript:
Note that I’m still using the “javascripts” parent folder, but underneath of that I’m organizing by functional area of the application. In the root “javascripts” folder, are the primary application files – the ones that define the overall application bits. In the “mail” folder are all of the files that relate to the “mail” application. And, in the “contacts” folder are all of the files that relate to the “contacts” application.
I don’t care what “type” is contained in the file. That’s a completely irrelevant way to organize files to me. It makes no sense for me to organize files this way because many of my files contain more than one “type” of object. For example, I often put very simple model, collection and view definitions that are very closely related, in the same file.
Why Type-Based Folders Might Be A Good Idea
In spite of all my over-opinionated hand-waiving above, there are some good reasons to use type based folders in JavaScript. One reason is asynchronous file loading based on conventions.
You might have a JavaScript app that makes use of a templating engine (of which there are dozens, these days). It’s not always a good idea to pre-load every possible template in to the user’s browser, for download size and performance reasons. Sometimes it makes sense to fetch the template that you want the first time it’s requested.
To do this, it might make sense to use a convention to retrieve the files. I’ve seen several Backbone apps that use a jQuery selector to load files, as one example of this. When a Backbone view specifies a template as “#my-view-template”, the application’s template manager would make a request to the server to load something along the lines of “/templates/my-view-template.html”.
If you’re trying to organize your templates in a functional area of the application, you’ll have the added overhead of inserting the functional area folder name, such as “/mail/templates/inbox-template.html” for an “Inbox” view in a “mail” app, trying to load an “#inbox-template” jQuery selector as the template to render.
So… there’s at least one possible reason to use a type-based folder. I would still stick the type-based folder name under my functional area, though. I don’t want to mix up the templates between my functional areas of the application, by accident.
A Tradeoff: Folder Names vs File Names
Here’s one potential trade-off for using functional folder names vs type based folder names: you might have to specify the type in the file name. For example, if you have a model type name “Person”, a collection type named “Persons”, and a view to represent a single person or a collection of persons, what do you call that view? If you’re organizing things by type, you can call every “Person” and “Persons”
This can be very confusing. I was recently working with a client who was using an editor that only shows the file name for the open files, and none of the folder path for the files. He ended up with 4 “person.js” files in his open file list. Which one was the Person model, view, router, or controller file? We had to open each file to find the one we needed, every time. So, we took the hit on the file names. The “person.js” file contained the person model, while “persons.js” contained the collection, “personviews.js” contained the view definitions and “personrouter.js” contained the router. Thus, we’ve moved the need for specifying the object type from the folder structure (with all it’s bad ideas for using that) to the file name.
I’ve read at least one blog post that advocated using type-based folder names specifically to avoid the “ugliness” of having type names in your files. I seriously laughed out loud when I read that. Whether the type is in the file name or folder name is a moot point. You’re likely going to end up specifying the type somewhere. I would much rather have it in my file names because it’s easier for me to see things grouped together based on functionality, than based on the type of object contained in a file.
It’s Just An Opinion, And A Loose One At That
My own use of these conventions and ideas is rather loose at this point. I don’t stick strictly to anything, and I mix and match based on the project type, number of files and other constraints that a given project presents. Because, like I said at the top of this egregiously long post, you’re doing it right.
No matter what file and folder structure you pick for you JavaScript apps (assuming you’re using a suite of libraries that doesn’t force you in to a specific folder structure), you’re doing it right. JavaScript in a browser environment really doesn’t care what the file and folder structure is. But that doesn’t mean we as developers shouldn’t care. Pick a file and folder structure that fits the constraints of your application and change the structure as your app’s constraints change.
Earlier this month I had the opportunity to record an episode of Herding Code with GitHubbers Paul Betts and Phil Haack on the state of Git for Windows: Herding Code 132. Topics included discussions on why Git hates developers and why Mercurial proponents’ claim that Git doesn’t preserve history is “nonsense.” I think the show turned out pretty well, and we got some hints as to what we can expect from GitHub for Windows. Thanks to the Herding Code guys for having us on!
Speaking of posh-git, there are a bunch of new goodies in my master branch including tab expansion for remote branches, support for the Nuget console, and easier integration with posh-hg. I’ll post more details with the upcoming v0.4 release.
In one of my current apps for a client, I have an activity based security system that determines what the user is allowed to do. The trick to this system is that all of the authorization checks happen on the server, but the functionality that is being secured runs on the client, in JavaScript.
This is a bit of a problem. If I send all of the JavaScript that is uses to run any part of the system to the browser, then it’s possible that a clever user could enable it and do things they aren’t supposed to do. Of course, when they send a request back to the server, the server will verify they can do what they requested and block it… but the user should not be able to even try to do things they aren’t authorized to do, in the first place.
The solution that I’ve come up with, at a very high “functional area of the application” level, is simply not to send JavaScript to the browser if the user is not allowed to use it. This keeps me from having to do extra security checks in the browser, keeps the download for the user smaller and generally makes the app appear snappier because there is less code to run. More importantly, though, it keeps the user from being able to try things they can’t do.
Modules To The Rescue
I’m facilitating this through the use of modules in my JavaScript – both “module” as in the JavaScript module pattern, and “module” as in a packaged functional area of an application.
It’s worth noting, though, that I’m not using AMD (asynchronous module definitions), RequireJS, or any other “module” framework for JavaScript. The debate about the “right way” to do modules can rage on all it wants. I’m content with simple immediate functions and logical groupings of files while the more intelligent and invested people in the JS community figure that all out.
Each of the functional areas of my application is built in it’s own set of files. Each area has multiple files that it is composed within, but the files are grouped together for easy identification of what makes up a module. I also have an HTML template for each module which provides the needed <script> tags to include the code for that module.
Only Render What Is Needed
When I need a functional area of my site to be sent down to the user, I tell my server side template language to include the correct HTML file. For example, I’m doing this in an ASP.NET MVC application:
@{var user = (CustomPrincipal)User;}
@if (user.Can("locations/manage"))
{
@Html.LocationManagementScripts()
}
@if (user.Can("locations/search"))
{
@Html.LocationSearchScripts()
}
In this code, I’m checking to see if the current user is allowed to manage locations. If they are, the extension method “LocationManagementScripts” is called. This in turn renders the “LocationManagementScript.html” file at this point in my HTML layout. That file contains all of the <script> tags for the location management JavaScript app. In the same way, I’m checking to see if the user can search through locations, and running the same basic process if they can.
Self-Initializing Modules
When a functional module is included after passing one of these checks, it needs a way to get itself spun up and started so that it can do it’s magic. It may need to render something on to the screen. It may need to register itself with the application’s event aggregator, or any of a number of other things. This is where my Backbone.Marionette add-on comes in to play for my Backbone apps.
Marionette has an explicit concept of an “initializer” tied to it’s Application objects. When you create an instance of an Application object, you can call “app.addInitializer” and pass a callback function. The callback function represents everything that your module needs to do, to get itself up and running. All of these initializer functions – no matter how many you add – get fired when you call “app.start()”.
myApp = new Backbone.Marionette.Application();
myApp.addInitializer(function(options){
var myView = new MyView({
model: options.someModel
});
MyApp.mainRegion.show(myView);
});
myApp.addInitializer(function(options){
new MyRouter();
new SomeOtherView().render();
});
myApp.start();
Each functional area of my application has it’s own initializer function. When a functional area has been included in the rendered <script> tags, the initializer gets added and when the “start” method is called, the modules for that functional area are fired up and they do there thing.
A Composite App, And Sub-Apps
One of the tricks to making all of this work, is that I need to have a primary “app” object that all of my modules know about. In the above example, the “myApp” object is this. Each of the modules for each of the functional areas has direct knowledge of this object and can call public APIs on it – including the “addInitializer” method.
A better example of what a module definition and initializer might look like, would be this:
// --------------------
// app.js
myApp = new Backbone.Marionette.Application();
myApp.addRegions({
mainRegion: "#main"
});
// --------------------
// locationSearch.js
(function(myApp, Backbone){
var SearchView = Backbone.View.extend({
// ...
});
myApp.addRegions({
searchRegion: "#search"
});
myApp.addInitializer(function(){
var view = new SearchView();
view.render();
myApp.searchRegion.show(view);
});
})(myApp, Backbone);
// --------------------
// index.html
<script language="javascript">
myApp.start();
</script>
In this example, I’m using the simple JavaScript module pattern to encapsulate my search functionality. I’m also providing an initializer for the module that instantiates a search view and shows it to the user using a region manager.
Each of these functional areas is basically a sub-application. Many sub-applications are used to compose a larger application and overall experience for the user. The composition of a larger application through various modules that are included / excluded based on some criteria are what really make this a composite application.
I also included the final call to “myApp.start()”, showing that I do this from my main HTML page and not from my JavaScript files. This provides a single point of entry for all of the registered modules, no matter which modules are registered. The “myApp” object really doesn’t care which modules are registered, honestly. It doesn’t need to care. It only needs to execute the initializers that happen to be present. If none are present because the user didn’t have permission to do anything, then nothing happens when this method is called and the user won’t see anything.
Security: Don’t Let Them See It If They Can’t Do It
If the security check to see if the user is allowed to use the location search feature fails, the rendered HTML won’t include the <script> tags for the “locationSearch.js” file. If this file is not sent down to the browser, then it will never register itself. If a module has not registered itself for initialization, it’s views won’t show up on the screen and the user won’t be able to try and use the feature. Further, the user won’t be able to “view source” on the page and find any stray JavaScript that they shouldn’t be able to use.
It’s Not Always That Easy
Of course there are other security concerns that are not this simple. When a functional area is closed off by authorization, it’s easy to keep things clean like this. We can compose the application at run time simply by including the right files and letting the code in those files register themselves for initialization. But when we have a functional area of the system that has finer grained authorization and permissions associated with it, things get a little more tricky.
I’m still learning and exploring this space. I have some ideas and am going to be implementing some of them soon. If anyone out there has any experience in handling finer grained security needs in JavaScript apps, I’d love to hear about it. Post links to your favorite resources for this, in the comments.