Scaling tracking and analytics

 Modernising and Scaling the frontend

## What is it

If you have ever seen something like `O(n)` or `O(2n)` when looking at some algorithm, like a sorting algorithm, for example, that was __Big O__ notation.

It's a way of describing or representing an algorithm's efficiency.
How long is takes to run or how much memory it occupies (time or space efficiency).

__Big O__ is a very useful concept for any programmer but is often neglected or not very well understood, by either programmers with degrees or those self-taught.

Understanding it will give you not only a better idea of how the most popular algorithms behave and perform but, more importantly: __You will be able to better judge how good or bad your own algorithms can be__.

And remember an algorithm could mean any particular piece of code you write to solve a particular problem, not just those you read online about sorting or searching things.

## What it means

Take `O(1)` for instance:

- The `O` is the convention. It means... __Big O__.
- The stuff inside the parentheses `(1)` represents how the algorithm __scales__ with its input. 

Here, it tells us that this algorithm scale is `1` or constant, no matter how big the input gets, it always takes the same time.

Here's a silly example:

```javascript
let array = [1,2,3,4];

const get_array_at = (array = [], index = 0) => { 
 //step to check for bounds
 if(index <= array.length-1) { 
 //step to get the array index
 return array[index]; 
 }
 //step to return
 return null;
}
```

We don't know about the machine we are running this on or the underlying implementation of the array, so we don't take those into account. We are describing *our* algorithm

How long does it take to run our `get_array_at` if the array had `10` elements? What about `1000000000` elements?

Isn't it the same? we always do the same number of operations regardless of the size of the array. No matter how big our array is our algorithm always does the same number of steps.

Therefore our `get_array_at` scale is constant - `O(1)`

Let's look at another example: `O(n)`:

```javascript
let array = [1,2,3,4];

const print_array = (array = []) => {
 let n = array.length;
 for(var i=0; i<=n; i++) { 
 //n operations: one for each element o the array
 console.log(`array idx:${i} has value ${array[i]}`); 
 }
}
```

What about `print_array`? how long does it take to run with `10` items? what about `1000000000` items?

The more elements we have, the more operations it takes. 
Our algorithm takes longer the bigger its input `n` is.

If we were to graph the number of operations our `O(n)` algorithm would do, depending on the size of the input, it would look something like this:

<img src="http://i.imgur.com/pFFsUvr.png" style="max-height:300px" />


As we can see, this is a linear graph. 
`O(n)` means linear time. (or space if that is what we are measuring).

Here are a few others that are most common: 
`O(1)` is constant. No matter how big the input, always takes the same time. 
`O(N^2)` (N^2 is the same as N2) is quadratic time. 
`O(2^N)` is exponential time. 
`O(log N)` is logarithmic time.

Here's how they look if we were to graph them:


<img src="https://i.stack.imgur.com/8nXvk.jpg" style="max-height:300px" /> 
<lengend>Big O graph found in the wild</legend>


As we can see, some are definitely better than others.
No one, if they could help it, would want to have an algorithm have `O(n^n)` runtime efficiency.

There is no fixed list of possible `O(..)`.

## How Big O is measured

*Hope for the best. Prepare for the worst.*

As you've seen, Big O is a measurement. However, it is not meant to be scientifically precise. It would be very hard to find the exact `O(..)` of more complex algorithms. 

In fact an algorithm can have different run times in particular scenarios or edge cases. 
Instead, it is an estimate, generally accounting for the expected possible scenario, or most common...

For example, consider sorting. 
Generally, if we try to sort an array that is already sorted, that is considered the best-case scenario, or if the array is completely reversed, that would be the worst-case scenario, where it would take the longest runtime. 

Some other more complex algorithms may take orders of magnitude longer than their expected case. In most cases, though, the worst possible scenario is the same as the most common or expected one.

In order to simplify, __Big O describes the expected scenario.__ 
That means, we know that an algorithm, while it may take less time, it will __never__ take longer than its Big O.

__The Big in Big O is about Big numbers__

As previously said, Big O reflects how an algorithm's efficiency __scales__, and by scale, we mean a big one. 

This means that if we put really big numbers into a Big O representation with more than one term, we can often simplify it.

Let's look at two examples:

### Removing constants

Something like `O(2n)` can be simplified as `O(n)`, by dropping the constant. 
If we think about it, when using really large `n` numbers, the `2` will have little or almost no impact on the overall performance. If you remove 2 pretty much the same.

### Dropping smaller terms

What if we have something like `O(2*2^n + 10*n)` or `O(n + n^2)`? 
The same rule applies. For really big numbers of `n` the bigger term will become far more impactful than all the others. We can drop the other ones and simplify it.

Remember, __Big O describes how an algorithm's efficiency scales with its input.__

`O(n*2^n + 10*n)`, the biggest factor is the exponent, so we keep that, in terms of scale it is essentially `O(2^n)`. 
`O(n + n^2)` can be the same as `O(n^2)` 

That's why, generally, most algorithms can be expressed using the most common notation in the graph above. Because we can simplify a lot of the cases.

There is more that can be said about Big O. 
To correctly use it for your own algorithms, or other people's, it takes some practice. 
However just having a general idea of what it is and what the most common Big O's are, goes a long way to demystifying it.

Nest time you run into one of these, or someone mentions an "O of something" in a conversation, you will know.

What the code is Big O

Over the past few weeks I've working with the [Aurelia framework](http://aurelia.io/), and I've been liking it quite a lot. 
The framework is quite stable and has RC release quickly approaching!

If you're trying to use aurelia for something more serious you will most likely be wondering about best pratices and project structure and etc.

This is an area I still find aurelia lacking, as the docs are very slim and there is not a lot of documentation out there.
There is a skeleton you can get started with, but it's not very complete. *it's a skeleton after all.*

Fortunately there is more and more community activity around topics like this. 
Aurelia is also a very well designed framework, which makes it very flexible.

Since in you can define everything in your app as a component, it's easy to break you app in multiple components or modules, depending on your needs.

As of now, you can use start using Aurelia in a few different ways. I am using aurelia with wepack and ES6. 
Naturally, I started with [aurelia's webpack+es6 skeleton](https://github.com/aurelia/skeleton-navigation/tree/master/skeleton-es2016-webpack).

This is how I ended up structuring my aurelia app:
 


#### Code Structure
 This is the code structure based on the current Aurelia best pratices and our own needs. 
 
 - `/docs` - docs generated by esdoc - *see esdoc*
 - `/locale` - List of translation files - *see i18next*
 - `/[en|pt|etc]/translation.json`
 - `/src`
 - `/configs` - *configs files for app, plugin, etc*
 - `/dev`
 - `/prod`
 - `/img`
 - `/styles`
 - `/components`
 - `/elements` - *reusable components*
 - `/value-converters`
 - `/attributes`
 - `/features` - *contains 'aurelia features'*
 - `/lib` - *3rd party code not coming from node_modules for w/e reason*
 - `/pages` - *all pages of the app that match a route*
 - `/services` - *app domains logic lives inside a service*
 - `/shells` - *app shells/entry points*
 - `main.js` - **entry point of the app. bootstraps aurelia**
 - `/styleguide`
 - `/lint` - *linting rules*
 - `/tests` 

 
 

As you can see the structure is quite similiar to aurelia's peer freameworks. 
There just a few extra directories for types of components aurelia offers "out-of-box".
 

#### NPM Tasks
 
These are some of the npm tasks I have defined for my workflow:

```json
 "dev": "webpack-dev-server --config webpack.config.js --hot --inline --progress --devtool eval",
 "build": "webpack --config webpack.config.js --progress --profile",
 "dist": "webpack -p --config webpack.prod.config.js --progress --devtool source-map",
 "cplang": "cp -R locale dist/",
 "deploy": "npm run dist && npm run cplang",
 "test": "karma start",
 "esdocs": "esdoc -c esdoc.json"
```

### Webpack config

These are the most relevant parts of the the webpack config I am using:

```javascript

//...
 new AureliaWebpackPlugin({
 includeSubModules: [
 { moduleId: 'aurelia-auth' },
 { moduleId: 'aurelia-i18n' }
 //..etc
 ]
 }),
 //...
 new DefinePlugin({
 __VERSION__: JSON.stringify(pkg.version),
 __ENV__: JSON.stringify('DEV')
 })
 ],
//...
 resolve: {
 alias: {
 components: srcPath + '/components/',
 services: srcPath + '/services/',
 features: srcPath + '/features/',
 utils: srcPath + '/utils/',
 images: srcPath + '/assets/img',
 styles: srcPath + '/assets/styles',
 lib: srcPath + '/lib',
 langs: srcPath + '/langs',
 configs: srcPath + '/configs',
 envconfigs: srcPath + '/configs/dev'
 }
 },
```

The prod is pretty much similar except for prod specific path/etc.

It's the best looking webpack cfg file, but I'm not an expert on Webpack, in fact I try to keep it a simple as possible, unlike a lot of config I see in the wild that get way to obtuse for my tastes. 

It's a config file after all, a little repetition for the sake of clarity is not the end of the world.



I find that with this struture I pretty much cover all the stuff I need.

It's a mix of the skeleton code, with some blog posts I read and my own preference.

Notes on Structuring aurelia apps

In the last semester of this year I got really into tech and programming podcasts and
ever since I've been always looking out for great podcasts to listen to.

Since I don't have time to listen to every podcast I would like, I have to trim my list to the best of the most
relevant topics, which in my case are Web, JavaScript and .NET related.

In retrospect this is my must-listen list of some of the best programming podcast for this new year.

 

#### - [Javascript Jabber](https://devchat.tv/js-jabber/)

Avg Duration: 50m - 1h 
Every Wednesday a few hosts talk about a Javascript topic, be it a popular library
or a popular topic in the JS community.

One of my personal favorites. It's hosted by a few people and usually a guest, the show is lively and the dialogue between the hosts
is interesting even on episodes about frameworks/libaries that I don't necessarily care for. 
It is a bit too lengthy for the stuff they talk about sometimes.

If you're into JS this is one is a must.

 

#### - [FiveJS](https://fivejs.codeschool.com/)

Avg Duration: 5-9min 
Every Thursday a condensed talk about the most recent JS topics.

Great for short periods when I'm free, It's fast paced and covers a wide range of topics. 
The hosts rotate between episodes and the conversation between the hosts feels a bit awkward at times,
as if they're reading their dialogue off a script or teleprompt. 
The topics are interesting but because of time they skim over some of them.

If you're just looking to stay on stop of the some of the stuff happening in
the JS world this one is perfect.

 

#### - [.NET Rocks!](https://www.dotnetrocks.com/)

Avg Dur: 50min 
Every Tuesday, Wednesday and Thursday an episode about the .NET world and programming
in general, like libraries, methodologies, patterns or tools.

My second favorite podcast. A bit lengthy, but the two hosts are great, and so are the guests. 
It's slow paced, at times very much so. The discussion and topics are super interesting, but sometimes the hosts
wander off completely off topic for too long.

If .NET is something you're interested in this is a must.

 

#### - [CoderRadio](http://www.jupiterbroadcasting.com/show/coderradio/)
Avg Duration: 50m - 1h 
Every Monday, a chat about what is happening in the world of Software Dev. 
A show not so much about a framework or language, but about our industry.

This one either I really like or not really. It's quite lengthy but the hosts keep the discussing very alive, but also tend
to go off topic, the topics themselves are interesting most of the times. 
Some shows are a bit too ranty, when the hosts go on long rants about X or Y for long enough to become annoying for me.

If you like to listen to some discussion about the industry, you'll love this one.

 

#### - [CodingBlocks](http://www.codingblocks.net/)
Avg Duration: 1:10h - 1.40h 
Monthly/Bimonthly talk about a programming topic like design patterns, best practices, stacks,
paradigms or tools, sometimes a specific framework or technique (most often in .NET or JS but not only)

A bit lengthy, and sometimes a bit too slow paced. The three host are very good, but the show takes a while
before it actually goes into the actual topic, as the hosts go a little bit off topic. 
But I really like to hear their discussions, as it contains a lot of experienced and interesting opinions.

If you're interested in design patters, best practices or tools, this is a good one, specially if you're into .NET or JS.

 

#### - [Developer Tea](https://developertea.com)
Avg Duration: 15m - 30m 
Every three days, a talk and often interviews with advices for developers how to generally became a better developer,
best practices, good habits and some language talk.

I really enjoy some of the episodes of this one. 
Most episodes are advice or opinions on a specific topic, which are usually very good, or interviews, which I also enjoy. 
The guests are great but the host is a bit too mellow though (as if he's giving a sermon). 

If you're interested in hearing about other experienced developer's experiences and how to improve you craft, this is one is great.

 

#### - [The Web Platform](http://thewebplatform.libsyn.com/)
Avg Duration: 50m - 1:10h 
As far as I can tell, a bit irregular, usually on thursdays or tuesdays, but not every week.

I've only been recently listening to this one but I quite like it. The hosts and guests are very good and keep a good discussion going throughout. 
The show usually dives into a specific topic around the web, and covers it quite well giving some good information about it.

A great podcast for all those with an interest on the web.

 

#### - [Developer on Fire](http://developeronfire.com/)
Avg Duration: 40m - 1h 
Released every few days.

It's an interview podcast, where the host interviews an accomplished developer from the industry, about who they are, their work and their journey.

The more I listen to this podcast, the more I liked it. The host does a great job at interviewing the guests. 
It gives an invaluable insight into the lives of the guests, how they got to where they are, what they worked and are working on.

If you want to know more about the path and life of great developers, this podcast is a must.

 

There is a few others I listen to on and off, but there is only so much time in a day and unfortunately I can't listen to them all. 
It's a good problem to have though.

Best programming podcasts 2016

**Generators** are probably one of the lesser known, and most complex features of ES2015.

 

### What is a JS Generator anyway?

The basic idea is actually simple: Stop/pause a function execution at a specific point.

 

### How do I use them ?

Simple. 

Define a function with an `*`. Kinda like a pointers but not really. 
Inside the function, use the keyword `yeild` where you want the generator to be able to pause/stop. 
Call `next()` to move the generator to next step/yield. (kinda like a music playlist)

Here's an example.

```javascript

////this is a Generator function.
//notice the *, that's how you define it
function *generator(){

 //you then use the yield keyword to pause it
 yield "first stop";

 //another pause
 yield "second stop";

 //can be null too
 yield;

}

```

This is how you use it.

```javascript
//1. To use a generator you need something to control it.
// We call that an iterator.

//this is not a normal function call, you're not calling anything, you generating something you can then control, hence the name.
var iterator = generator();

// 2. then when you call .next, it will move on to next pause point, until it finishes
console.log( iterator.next() ); //logs Object {value: "first stop", done: false}
iterator.next(); //Object {value: "first stop", done: false}
iterator.next(); //Object {value: undefined, done: false}
```

As you can see, to use a generator, you define it, then create what we call an `iterator` (a variable, really), to control that generator.

As a plus, since we use an iterator to control the generator we can use ES2015's new [´for of´](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for...of) to make this task even easier:

```javascript
for (let it2 of generator()) {
 //notice
 //1. we don't have to call .next()
 //2. loop stops automatically once it's done
 console.log( it2 );
}
```

 

### What's the point?

Simple. This opens up a whole new slew of capabilities. 
One of the best is: *To handle asynchronous stuff in a synchronous looking manner.*

The trick lies in what `yield` can do for you. 
It doesn't simply pause the function execution.

The `yeild` can also **pass and receive** values from our generator.
Like a messenger between the generator and w/e is using it.

Here's an (stupid) example:

```javascript
function *playlist() {

 let scores = [];

 console.log(scores) //logs []
 let score = yield "Black rain";
 scores.push( score );

 console.log(scores) //logs [3]
 scores.push(yield "Trap door");

 console.log(scores) //[3, 5]

}

let it = playlist();

it.next(1); //Object {value: "Black rain", done: false}
it.next(3); //Object {value: "Trap door", done: false}
it.next(5); //Object {value: undefined, done: true}
```
See what's happening?

I'm getting the values from the yields, like the first example. 
I'm passing values into the generator, like this `.next(3)`. 

If you notice the `next(1)`, the 1 I passed was ignored because there was no yield to receive it, it was the first pause/stop.

The execution behavior may seem a little strange. That's because it is. 
That's how generators behave.

You can also nest generators inside each other using `yield`.

Now, all of this is still synchronous, but with this few capabilities we can start to go async.

 

### Where is the Asynchronous

Basically, we can use the generator to pause on a async operation and then resume, via `next()` at the end of that operation.

*Also, do you know about JS [promises](https://github.com/getify/You-Dont-Know-JS/blob/master/async%20&%20performance/ch3.md) already? If not, you there is not much to gain by learning about generators before that.*


Here's an example:


```javascript
//our async fn, it could be for ex an ajax request
function asyncSum( num1, num2, cb ) {
 setTimeout( () => {
 cb( num1 + num2 );
 }, 1000);
}

//a wrapper to abstract the callback away from the generator
function doSum( num1, num2 ) {
 asyncSum( num1, num2, function( result ) {
 it.next( result )
 });
}

function *sumGenerator() {
 let sum1 = yield doSum(1,1);
 	console.log( sum1 ); //logs 2 after 1000ms
 let sum2 = yield doSum(5,5);
 	console.log( sum2 ); //logs 10 after 2000ms
}

let it = sumGenerator();

//start our generator
it.next();

//1000sec later -> logs 2
//2000sec later -> logs 10

```

Even though our ``sumGenerator`` looks synchronous, and very simple, it is actually performing an asynchronous task. 
Our ``doSum`` function is being called by the generator, which then pauses. 
Once our ``doSum``'s ``callback`` is fired it calls ``.next()`` passing the result back into the generator which keeps resumes it's execution.

This does look nicer then nested callbacks. But is not that useful. 

All we're doing is turning asynchronous code into synchronous-esque code by **pausing our generator when starting an async task and only resuming once the that task is finshed**.


We don't have a lot of flexibility like this, since we can't easily do error handling, we don't have control over the generator once it starts and we don't have an easy way of actually making two asynchronous operations at once, because the generator pauses at each one.

But, aligned with ``promises``, they can make some really great thing make JS a lot more asynchronous friendly.

Here's where thing get powerful and interesting. And complex.

 

### Generators and Promises

The idea in again, simple. 
Just `yield` out a ``promise``.

We do that with two simples changes:

```javascript
function asyncSum( num1, num2, cb ) {
 setTimeout( () => {
 cb( num1 + num2 );
 }, 1000);
}

//tell our wrapper to return a promise
function doSum( num1, num2 ) {
 ////CHANGE NUMBER 1
 return new Promise( ( resolve, reject ) => {
 asyncSum( num1, num2, resolve );
 })
}

function *sumGenerator() {
 let sum1 = yield doSum(1,1);
 	console.log( sum1 ); //logs 2 after 1000ms
 let sum2 = yield doSum(5,5);
 	console.log( sum2 ); //logs 10 after 2000ms
}

let it = sumGenerator();

//start our generator, and now we have a promise yielded out
let p = it.next();

////CHANGE NUMBER 2
//that will resolve once the async code is done
p.value.then( ( result ) => {
 let p2 = it.next( result ).value;

 p2.then( (result2) => {
 it.next( result2 )
 })
 })
```
That's it, using promises, we have now control of our generator back.

However as you might have noticed, this will quickly get messy, especially in cases where you have multiple ``yields`` and have to do error handling.

No to mention, in some of the cases for asynchronous code, like ajax requests, you might want to run the some asynchronous tasks without stopping for each other to finish, as they might be independent.

So, to make this work properly we need some setup/boilerplate code. 
This where thing get a **little complex**.


Here's one o the many ways we could setup a helper function to do this for us:


```javascript
function asyncSum( num1, num2, cb ) {
 setTimeout( () => {
 cb( num1 + num2 );
 }, 1000);
}

//a wrapper to abstract the callback away from the generator
function doSum( num1, num2 ) {
 return new Promise( ( resolve, reject ) => {
 asyncSum( num1, num2, resolve );
 })
}

function *sumGenerator() {
 let sum1 = yield doSum(1,1);
 console.log(sum1);
 let sum2 = yield doSum(5,5);
 console.log(sum2);
}

function startGen( gen ) {
 let it = gen();
 let res;

 // return a promise for the generator to complete
 return Promise.resolve()
 .then( function goNext( val ){
 res = it.next( val );

 return (function onRes( res ){
 // generator is finished ?
 if ( res.done )
 return res.value;
 else {
 return Promise.resolve( res.value ).then(
 goNext, //go to the next yield once this promise is resolved
 // if the promise was rejected (an error occurred on the task), throw the erroe on the iterator
 (err) => {
 return Promise.resolve(
 it.throw( err )
 )
 .then( onRes ); 
 }
 );
 }
 })(res);
 } );
}

let p = startGen( sumGenerator );
// p is a promise that will fulfill once the generator completes
```

As you can see, this is neither obvious or simple, you need some time to figure out what it's doing. 
Basically, it is trying to run the a generator to completion, while checking if a ``yielded`` promise failed, if so, pass it up to the generator so he can handle it.

Of course this is also a minimal example, real world scenarios will need a lot more wiring to handle more complex cases, and if you're like me, no way you want to have to deal with that.


Luckily, like everything in the JS world, **there's a library for that**.
( [asynquence ](https://github.com/getify/asynquence) / [Q](https://github.com/kriskowal/q) )

And, ultimately ES7 is going to have another [feature](https://jakearchibald.com/2014/es7-async-functions/) , built on top of this one that abstracts all of this for you and make the whole thing better. With ``async`` and ``await`` keywords. 
Which works just like it does in ``C#``, if you are familiar with it.

 

### Why should I care?

Honestly? I would risk say, not a whole lot.

As I see it, Generators are a lower level pattern, used a building block for higher, and easier patterns. 
The 99% of us will be using an abstraction of it. Either with an helper library or with ES7's ``async & await``, which abstracts all this boilerplate and make this whole thing a lot easier to manage.

Notes on software

17 Jun, 2019Medium.com //Scaling tracking and analytics

29 May, 2019Medium.com // Modernising and Scaling the frontend

1 Jan, 2017What the code is Big O

28 Apr, 2016Notes on Structuring aurelia apps

1 Jan, 2016Best programming podcasts 2016

1 Dec, 2015What the code are JS Generators