. the . everybody I'm Joe some of those missing and are a group of Brooklyn Michigan to those of you don't know or that is that's where the dry erase track is off . I drove in a cornfield Owner Eastern and that's why Alicante or per hour delay had nothing else to do . so my career I started out small computer from laptops sailing the seas on laptops in an Arbor Michigan . I ended up working for the very first e cards lay on the year to alter two during the first movie . um . he checks the future maybe two so I decided comebacks this can finish up my degree at the glorious Easter Mr University . after that one out Seattle where we sticky notes which some of you may or may not in use in college it's one larger on the larger cut of up study guide and reference material sites . I worked there for awhile before getting hired at Digg . last Digg about . lower your data goes from a company called simple geo . we Build basically location infrastructure McLeod com . I want to talk about a couple different things one I'm obviously not serve the company around and very passionate about location of mobile radar . so one of the Kala bit about that . and how location is actually going to have a pretty negative impact on the scaling problems that we've been working on for a while . and then not . the last thing I really want our daughters and steal lean . scaling to me is not just stealing your infrastructure it's also scaling your teams in your processes . is this ongoing passion about engineering culture and how to get back to that lie within an organization economy fix for . so those of the things on the talk about um . he Backlog like Raymond Wyoming . until someone Twitter . from Korea like twenty care . I really the most important point here is that you shouldn't . I'm Just a random guy . so whether things that really Story to tell people what talk is ideally talk about my experiences and really stories and backgrounds and you know Wolf but what I've been through . um you could look at some stuff on this and think thoughts on that idea It's Miller stuff you might think is total bullshit which to live five Small Kazaa might be in your context . so starting off with location . so I think this number though I did they teach me in an I phone but nevermind . Nokia and Windows phone seven almost as the market . so you'll forgive the thousand odd loans today . you might think that whole what's the context around that number why are people so excited about mobile . to put that a little bit in the context . our silly o forty thousand laptops today than two dozen on . so the adoption of mobile on the talk about this the second is is quickly OKC the adoption of . we would consider to scope across to this point . but the good will that ever Context ears to billion users on the Internet . roughly twenty percent of the world population . as five billion mobile subscribers in the world . seventy one percent a world population . recently spoke at a conference in Africa . um . they're skipping laptops and computers there they're only divisive day out . is the mobile phone . palm also the really scary statistic that . in some suds there to African countries they spend fifty percent of their disposal and come on communication . um . so anything about it so we basically know we women during also lot about the sub ninety nine dollars in Durham said . so the adoption of these five billion mobile subscribers is Going accelerating very quickly towards Mobile smartphones essential . um this the kind of surprise me as someone of that . as the old large systems before . the series phones has roughly about six answers as we get an MC in a couple of things that'll probably expand over the next . I'm guessing twelve twenty four months or more like sensors . our . and of course phones are always on and are always reporting they're always shocking narrow is feeding information back . and they're much more ubiquitous . so one of the things that I did why actually . oh Low backstory Support you actually start of the mobile TV company Call crass for Conduct like that . um . and I've I did some back in and out the map what if we add ten million active key sessions a day . we had and location is a very integral parts all those who those people need to be phoning home on irregular basis and feel I did . so aside . I really started thinking our lab in Swan we track everywhere we go and everything we do in real time . particularly when you do that job the phone edit there a minimum something you'd want to track this what's a latitude wanted to the time stamp on which is really the bare minimum a you can actually try to Map phoned to track him to noisy to track them be a lie to track . which direction the point in this also depends to track along with that . so the back of the map in exercise I did this just to verify that I should be scared this I decided . what is Facebook . Who at the time had five hundred million active users decided that they wanted to try where and when those people work and they were only reporting home every thirty seconds . . and this is a violent crime I can't see any about thirty seven gigabytes per minute to storage . um . when six I might Senior . which for some two hundred and thirty g a month in the St cos honestly when Story thereby one of the dollar figure on it . . so this is basically what I said I one guy Cycle faeries like no we can do social gaming can do it is amazing the to do some intuitive for me to edit into of Focus infrastructure company which militants all my nightmares . um . something that also I think it is has already gotten really bad and is only get worse under these contacts . is an Irishman said this recently at a conference and really scare me every two days we create as much information as we did that on civilization till two thousand three . we had the Internet for ten years when this when we're in our reading that every two days . on the talk to people like this the cost of the goddess store scale and manage this baby that is no longer a problem for the big guys . if anybody has alike as anybody click of a garden hose on Twitter . Garn knows I did you get up to the garden hose lower uses accepted TOS I think that's pushing Automate into the second of Text . and that's if you like wow really bored onus find out what people are treating about you know around what's going on in User parade . all you'd need something that could sustain the mega been a band with him something that are sent like you're already edit the data promises live one . so put this in the context . we're creating two point six per two point six million to buy today today . keep in mind we still have alot of three billion person got in between . we're creating Lead now one what we're going to be creative mobile phones . . as I could push it . . so mom decided that since you . I really think that you know we've been bickering and moaning about the things a bit I did User nor does scale the real time weather . on the it compare is starting to feel the ghosts and warned of more to run into with the real time mobile web . so what is going about dot com . the first one I want to make on scaling and people really of our dominance This Keeling does not equal performance . so performances how about something operates in scaling is about how much it can handle . so that goes into never metrics so if you How Developers LD that are already are shots that are like really really worried about Double quotes Most Agile coats and PHP to Tehran pontoon . um that doesn't affect the ability to steal the website . audio They May offering a hearing shots two Feature Gobi plenty of this going on lean over two years . um PHP socks . you know why must we be stopped . I want Python . when reality . this is that this is a chart I didn't are deceived but the sickly the green User routine and the Blues PHP is the hunger it this is a hunter and iterations of requesting my site's RSS Team parsing XML . and that advocates eating out two ago the baseline write about here . Ruby is actually a lot later is far is the baseline goes . roughly about this and a modest the armor of these only a faster so the whole argument that PHP is faster than Ruby or whatever is better than whatever is doesn't really apply up to clean off the guilt lies two web scripting and grey web apps . what does apply . Israel Testable sweet memory like this that you would be right back and the men and Ruby are PHP . he dreaded something like Job our senior dot com . so really what it comes down to per ticket winner scaling large infrastructure . I'd like hearing Team almost every single Owner on to run into and scaling let's label Breakdown two audio it will be there . CTO I Old daylight . to drive Business Dio Network Iowa to walk on down to some Ohio problem . I'm Just a kind about the temple of it . com . this week measure based limited and that last Slide tied Network I over CTO . and that our dialogue PHP basically took up everything and very small portion her of the majority this is well . or as my buddy John Anderson said Sue is a former director Build flicker KEY Shias in scale . these show people This slide only tell you something stupid About one language Team fastened the other . so what is ailing . scaling is socialization . Mr Slide seventy is our bed most of Wednesday evening out the awful the will to we're doing . because the fact is what I had you in order to steal did you know it's another sites like that . . are probably not Double Gold's away you're doing on the Whitehouse and some very common schools and tricks of the trade five . does anyone to directly relate to you a different out the different of the departement . um a good example this is it is we are building our own geospatial been a sensible jail . but literally building day-to-day software so that we can meet the performance requirements of our customers . I'm fairly certain nobody else here would be writing geospatial data bases in order to satisfy the requirements that were there early on but there might be some things that you we do a better different one . some of the schools on . I'm going to school click to move schools a go you we use to be focusing on we still Automation on queries Inception Papa and or puppet . the next and next time I come than a lot more hands on everything should be automated particular yes you move into the cloud to cloud is really really right for Automation Amin . Amazon Amazon because it's actually really scary how quickly can spend thousands of dollars on Amazon just API call . . we've got mad at somebody is we've actually combine that with the aid of us guys . so I tell my guys are one my guys notice that say we do that and go to the Cosatu Gloucester . they essentially go and play or make API request that notice it boots up in our case am I the sickly phones On says . where's my Papa Master . it tells the words that and then They considers itself to the point where can be put into the cluster will email my guys you anything the entire process takes between five and maybe ten minutes from in the key to that is Automation the way I'm able to maintain the infrastructure is a BIG as we do with this few people is we do this for Automation . so if you're running shops that are maintaining more than . I would argue one server . she does when you look in the shelf and up into Automate that infrastructure . raising questions are asked mobile to use . do so . another thing that's I think a key aspects to scaling it is the notion of stealing however scaling up from thirty seven signals a big fan of the scaling up just for a bigger better hardware added which is fine if your User dots and in your skin your growth is scaling in almost one year fashion with Moore's law . if it's not then you have to . to go to other things to do without mom . so and that's been fined for Saddam's the scaling out is the idea that basically instead of buying . faster CPU use more around faster harder eyes bigger stands that you get your architecture into position to where you can just add more boxes . Google was really kind of the . well I guess I really the first but they were the first a Really ul large scale commodity are to wear . um there a CIO said one may only have like . we had something like two ten thousand those in the clusters in the early two thousand and one day like two hundred nodes of the cluster Howard is truly dead . palm . and this is Become ability popular with the skill of websites where even some of the very very large websites when something goes wrong with the dome the cluster with relief . what Alan thrown out . this the new one . the other things that you Medium you A Team Agile eaters Partition your data . there's a police All much as single icicle server can do . we've actually forgotten scale rights died despite adding more more bicycle slaves . but what my sequel still there's a lot of kind of common patterns but there isn't really it off the shelf forty of my seat for you can purchase rights . . I think probably put her and Facebook are falling through the larger bicycle Partition instances out there . and that the idea purchasing a day . actually Med cash in on what you'd is basically like the fall now . but the idea there with that is when you purchase your data is going after your users on one server after users on the other server in an hour in the click sound data about four secrets that User died User overhear ideas are ideas over here . com which has been a very very common way the skill of those rights if you need to . . . choose your use queues actively blog a while . doesn't use of any time . a couple . cool ideas . which uses induced . to us . what's up . MSN one . Kutcher is there anything that a delay . don't make themselves . arm . so those are to Enterprise dot chances are options out there are prominent you know which is Remember Lang one of People use the Castro was great like Twitter that was going on . the idea during is that when you can make User your Offload is much of your processing to background queues . so they Your User gets basically the website back It's innocently possible contend that baptism is possible . . so there's a little awkward Example this image uploading on flicker goes into a Queue brand . and the euro me Lead easily understand that the image has been uploaded in the mail Offload at the image that you which then slices and eyes the Internet standard sizes on the way . so here User doesn't need to wait around Value Create seven different copies and sizes of an image for you can Offload exit you . to them immediately and then log on . he actually the surprise also how much of a . how much of a tolerance areas to queuing one two star using engine using a lot and actually is a very nice way to scale while large infrastructure . if I may users really don't care if some sun doesn't happen for five seconds . don't know how to the immediate you can offload them and nobody will cry too much . the other rather than to defuse is that they tend to be buried area last to pray . if you this huge spike in traffic like a sum User uploads . much of something to your API . and that's all happening in the queue . the rest of the User Experience is involved down by processing all that in real time in a process that you can fill up in and of having you know fifty thousand things and that's ok to take a minute to chop through but it least the rest of your processes are being backed up by waiting for whatever is happening on skis . . another thing I don't see why people don't do this it's Cache . antibodies you're just like with them and gets annoyed that was typical of one player . God . especially I really large place an ad they're doing like fidelity to month they had no CD and . they were using them Cache and the reason I'm and Testers Lake . it just looks really complicated . honestly can you read the documentation is to function call the Mickey click . casting is one of those things that kept him actually queuing now in my opinion are are are minimum Very sentry was really no reason at this point not to do Cache . other than extremely easy this month . um QA is getting closer killings eleven were difficult director Blaine around the casting of . but again I think the jury systems are so easy to consume in use at this point there's really no reason not to not to start utilizing all schools . so many other two about others like Federer for bigger Data obviously there's no reason the pulpit you shouldn't be your first like go to toilet Developers log files right . but there's nothing wrong with implementing a cue from day one . some may know that a lot of People to overlook them and I go I heard a lot of this feedback today . it actually takes a lot People to do stuff um do you call rotations the Asian Easy yelp . people the Maintain servers in the people the code in the offing to go out yesterday is that when you Code sumthing . when code something else . the first thing you Code it doesn't disappear . she may continue Maintain . the code of benefit for thing in Mexico the many names on the previous three things Take also much time as code in the next new thing . um . so this is something that . he steal the time . why is it eight . twenty engineers Maintain day or whatever . it's because there's a lot of food and a lot to me she's gotten need to be maintained . um I would say that I did we'd probably spends the vast majority of our time on the tests and was soon on Actual Feature development . com . I know that a number the Cal quoted He was that footage years than fifty percent of their time . I mean it's an internal tolls . so fifty percent of women and unit I missed out on stuff that users will never see . she has an Agile LP which is lots of new Feature a longtime . fall . so some of these I want to talk about oh unit in this particular portion as it is what I refer to the game the talk is hurting Task corner manages engineers . to call home all smiles the room . Herding Coutts is my favorite euphemism for many engineers . because it's just like herding ducks . um bail was on a why . anyone UAT lessons don't why I am an engineer a dollar one to eighteen eighty doesn't want to you . twenty mins around I know it's like I would love to go talk to the county Manager helix I want to do if you're counting told you that I like working on candles on the demand . . I'm fairly certain that that's not a problem and I In and other organizations having a Really fashion about Iran . I've Manage engineers . this for some reason and this is is is really my . my eyes my bow Product for some reason . engineering is the only train Perfection for the rest of the business organization killed it's completely appropriate to put Team non trained person in charge of those people . we're somewhat Dennis off with Aldershot in demand by someone who is never a dull I code . . the one finding of a colleague Owner McCallum says BIG and us . and finally County was like . no people around the county not a CPA that doesn't happen . does the minister I Q so insane . they made a lot talk a little while my thoughts and beliefs on building scaling kind of managing an engineering organizations . was I to say . going from small time . the time . although there's action less well one of the person your head . grilling is the to the engineers . I'd they're going to protest . every step of the way . I've never met a group of people who are working in industry were literally literally everything you do everything the entire corpus of your expert knowledge is useless . every two years . I . in yet . as a whole engineers tend to be Billy resistance to try to change . it doesn't make sense . so the first year to twenty like the ice with which is a broad cross Scrum Iris in the lake . I'm sorry I've got I've got the claimant that I don't like change . I just we rode the entire outcome Job and the note the J eyes but I don't like change . that and when and why is that high . Vision is a very interesting bunch to be stubborn . she to say this they one a lot . as a form or easier by wind of it on . why look back on what I mean the guy's like a living hell . little brother borrow it . lazy . twenty oh eight We was a totally easy . callers would not you would not be in good order one now lazy if you think about why you got into programming . a lot of People given up already because they find out they can Automate stuff include bikes automated stock . lazy people . I did not consider the scenario that's is an engineer is the one you never see your door here . that's because they've automated everything and the bottom is the point where you don't even have to worry about it . so I actually thought I actually look for engineers who are lazy I want to find the guy in that was like yeahh was doing the same words keeping track of whatever and I decided that I would write this like twenty one Python scripts in order to like Automate tells of the type of people I think they really to programmers . military of think anybody can argue with . we are eccentric . of the range of personalities A Iran into an engineer organization will range everything from the to do's to psychology once a code everything a mother in an indictment says . although it would a guy that was really cold on the two hundreds in the early eighties for banks . who doesn't one change anything Epic schoolgirls is fine . . so course a question is how you do it . I think the number one thing you should think about when you were creating an engineer aren't is a chance . they need to lower barriers . want to go local equate this with moving up the stock . um we should welcome the new engineering organization eyes and see what barriers they are two younger engineers performing at a higher level he should aggressively move through those barriers . I have ever . QUESTIONS . Split probably him two or three months after I started it . and appetite are needed only had . Riyadh the engineers would argue we Going to earth three of us . actually Kiribati an engineer . and JSC this eleven sign as he where I find more chilling new eyes was for the teams we were pretty much the two senior engineers their time . I said yes the wrong question . she one you don't wall one for me of Joseph was fun around the extensive fucker marginally I can on I exited every one of those traits I just showed up on the board . what you wanted you is your campus structure processes and code until place . read be going higher . hundred junior and mid Low adherence and they are efficient and are able to implement things quickly . Google's actually been very very good about this they have . all sorts of Frameworks and cool and everything else that are available to all of their engineers . this is actually a reason why when wanna go work it cool . all of the interesting part comes in Salt Lake . much fun is that when someone says oh you wanna processing of fourteen currently today the Nobel need this if this will API and point and fifty thousand or cluster will run over that never backed unit couple hours . it . They blowhard carriers of entry to the point where literally for processing something like a hundred years of video every twenty four hours just happens . I've been automated . so this is the thing that you should really be out this is the number one Team trio tell four . as I said . the reason for this is because Dio era . I'm fine he knows I have a guy that runs a very successful shopping CDA go . they're probably doing a couple million in revenue year and Andes having Oliver time finding a really Senior PHP program . com and I would argue that one of the reasons for that is because he's not a position to where you can hire programmers with Blitz LaSalle . so called things that I've done to help click to setup the house . for singing . I'm not a BIG banded and I don't think a lot people here are you . of the notion of the allied with what is the Holden Astra cure oh Program or the person that goes into the cave . codes for forty eight hours of the six pack Arrived bowling comes out with like this piece a code with no comments no unit tests . nobody knows how to maintain and no documentation . this is here . some big banner teams in particular a BIG Job cross functional teams . this is most Works continually watching an Agile shots . the old way of doing things . was . don't you add to the core infrastructure only did was work on this one piece . we had awesome Only duo's work on this one piece many had Friday Developers Lily work in this one piece . it was we wanted to build something you need a little bit of the to those things when you knew that something new decor for surgery edit your operation the second biggest things such and such . delivery teams use a Leo I did this and you have to push it through them to push it through them and it was a time not to bury . twenty in an apartment on a very efficient way to to basically Build features . so why prefer Easy and this is where you how a team that's working on Feature . and the Team consists of the appropriate . Coles has an People to implement those features . so I'm a very high like a psycho we govern our day Real watching Product symbols Hill . I have two of my core infrastructure guys I have one application at all for one operations guy and of one of these would Whole developer that's working on the school Product and . somehow the tools I need there without having to like mix and match an email CodeThe operations guides a wizened This Thomas Teams blocking on a separate center . it's a seizure due to a unit of the Team bits . something that endures . I don't like to do much outside of When Morris's Communicate arm . one of the window but Tucker we started at Digg . . saving . so a BIG piano of asynchronous communication I think that the peak Unknown Ross . the engineers sometimes have to do accommodated a basis . it can be extremely disruptive to edit to programming . um so why say Communicate I don't mean it costs only want engineers military of the dog . because you're basically in an interrupt the flow the infamous Florent . so some things I do around communication that is . I really had a really good job and some somewhat Said A combative about it about . I don't ever wanna see somebody was not an engineer walk up to an engineer Tom On shoulder to Ryan's for its mean all sports Update the day . but to but also in an image in addition put walls and and policies in place so that people don't feel that they can to give and years they just need to know that that the proper channel for reporting above is supported in the bugs the law not to walk over the engineer ability that some working on some of these that we have done with great success it is would you cab . basically um we did this the day and at school Job . we bought all of Thursday's snow methinks . we all come than the other thing I'd tell the people who want to have means of engineers . this year from of this are Backlog those on the day . don't don't give In engineers six meetings over an eight hour day the hour each . three hours the meeting of the DNA there were three hours means that in the day give them a BIG took the time to actually . Double four . I'm . another thing that is scarily absent from any Development shops standards and conventions . I was again no claim for Plan Warwick and are near silent PHP Ottawa again the Plan more about at the Python . I just want you to pick up walking horse and stick with it . all I was so alive . bill nicely About steers the conventions is usually there are some very smart people in your language of choice . Who have already done this work for you . every language Java has its standards which of course you know decree from on high . They has Benevolent dictator is that this is all we could Things and Python . I love about my son is there so pedantic about the concert coding conventions in force in the language . palm . yes so Double YY the ICU was with great success of a lot of People taiko standards . four use those . sometimes Build gold for the mum I think that's fine . when I enforce this is up for Your Users continues Integration . that's our . the COC of the palm . we are Not Noah continues Management . do for him to something I do and could use Integration that is and also in my code robot stories . as I put a link checker in there and Cardinal and checkered is also argued for cement of the coding standards which drives people and saying I don't care right CodeThe looks the same . so these are things I'm going to do . in importance and you really from the beginning . the committee can do it this financial the rowing for encoding goal socialist . this will call you Dumbo one flame or out of every Slide emotional . I really think code above the Tories . two dozen unit Obama Story . it was shot yesterday how many common that I window shop this is amazing for a new shop recently . the company is doing . two hundred and fifty million dollars in revenue . ninety five percent of which is what basis . they do not use a code repository . the ants . actually it's art . they just started using a code repository . he and the way of deploying a load is to basically razor Agile aka is ok but if I pushed wrong . and they go on the server and you see an update . um . does amazing . and I one I was talking with them they team up with it she was talking to them is actually two different teams operating this manner . in one of them is in a different building . similarly call somebody and say you yell around the office in Suva to give a push from can help the do just broke trying one out of the one out to lunch when they come in an AOL go on the reset Maintain summit was strong . and I did I see a part of their operations department has been begging them to do basically branching and tagging him pushing from from tax . and that he actually created a chart nice add on the seal was the number one countries like milk over by the door or one diligent code out there System Apostle the CEOs of this . and I saw Brian the I T guy I said . you're not doing Value undercover the planning are not using go deposit Tories and the one to socio how much that costs and downtime . and he said . the look of the CEOs business and yet operation said while to see the new temple we went on for forty five minutes at ten am last Tuesday which is basically KEY traffic key day . is like that costs us twenty eight thousand dollars . do you and any CTO I'd previously told me that any entity operates die so that the absence between five . you are between three and six times a month . between three and six all draw we're losing between fifteen hundred thousand dollars a month one to one point three million dollars year revenue of the albeit you are when you put that any zero that Remember It's impactful and it will make people change their attitudes Frequent critically when the patient numbers like that . the Tennessee I said you realize that you are wasting one engineer salary about every month . teams like ah um . they are not willing to source control Management . another thing but it is really boring is promoting ownership . this is really goes from top to bottom . it's really important for programmers . I really truly the programmers are craftsmen . we care about what we're doing We Carol building the code how it still . the problem for us is that nobody else cares how it still . nobody outside of engineers give a shit They Euro some awesome abstraction layer and and were able to condense that you know spaghetti code on into town once something is really important when engineers come to work for you you allow them to own the code . it's really not gorge on to dictate implementation details Feature Dodd to call bullshit when the telly that the one you something stupid like Ruby for demons are PHP for tenants . but it's not your doctor Really dictate that such a grand granular level . um so I tend to like to allow that the engineers at work for me to make those implementation decisions and to own the code from and and from day one until the end of its life cycle of the many cases all in and let them make the decision of one to pull something out . which is currently happening at simply overflowing in summing On replacement note . Double mother discussion . I'm . another thing but we often overlook is designed . I need people here Stub out the API is on a whiteboard before putting For on paper before reading the code . I like to see more and some al one . you the amazing go next Context I really made the right of free market some sort that is . They going to consist of more than three to five classes in more than twenty ish methods . go grab some way they respect their opinion go to the whiteboard say I need something that is going to do this and whiteboard it before you exit open the paper the to be impressed with how much easier the coding goes when you started on a whiteboard and how many problems that you can get over just piling up the API . and no don't need to write code assist with relief class limbs method names what they're taking . to what the returning nothing more nothing less . . our mother . when Use wakeboarding as a metaphor for the prototype . yes I know that it's more like Dom . I come unit I say I wanted Build buildings . for seniors Build blueprint Trend yet so it's more like trying blue friend . I think . um . so if the idea doesn't always to be a whiteboard can do it stubborn Old and then you can in paper or whatever . like snow white gorgeous . it's a swimmer . but making certain to cause lots of flame wars A office Frameworks . again I don't care if one could neither tango Wales . the shadow two . . . all in . . it's . he an hour . Polish . this is an investment . . she . one . wana say when you said setup which is just as good . yes are totally and completely rid the if you even if you Design on a whiteboard and a menu or write to us in you find you there are pieces of the day you can feel unit tests . you're in the IDC he knew to go back to read the whiteboard . so decide . on the talk talk more about us and so on . yes about to Frameworks and we will Frameworks at Digg is mainly because you can in the cold chain off the shelf . I actually did the best way I heard this quote was actually from a quarter Build Burke was talk about Wales where he said that in order to steal to edit or where they were they had to remove wiki called that much of this tactic sugar Wales . so yes says Twitter uses Wales but it looks awful I did when implementation details and the rails Code that that you would write and push a recruit . on the same goes for J don't Python same goes for coding I turned PHP and all the other Frameworks PHP . so what I ended up doing quantitative actually wrote . very small domain specific Frameworks Amin . her little Ajax framework in latex remarkable a day was you know check for CSR a Another exploits and with just a little PHP classes Map and do your I return and Reduce on the Qantas dot com . one of the reasons to roll your own other than massive scale is also . I've done this very successfully a couple shots . Who is coming to shop for work so the shop where they don't use and Use the mid to does open up a PHP files are reading the copy paste outcomes of who who works as Cover that don't have a framework . when one person . was an inside Frameworks Willis is Job answers Use ing like a dot net as framework of some sort . . yes it was using a custom frame work . knowing one of Wales . to touch and go . Unknown nudging an awesome coffee some cheesy for workers like helium . so one of things the dominant house with because it was totally day was literally going to the children trio PHP files second away whatever was includes functions out PHP which is literally like to keep the ones Code Amin . it was very easy for me their way Bad Ajax framework . and this kind of like it was so small Without PRT with no three was a code it's not that big a deal and actually explain I see Use . those your foot in the door . in the light Automate go online to find work elsewhere although application like Manila today the doors overall more than you keep going and XML . Porter The really over takes a while . since are easy prey Work summons and Works mum particular slide . seeing who here doesn't us the code . it's good for stretching . some of these around testing the radar . are often overlooked or confused by developers is that there is a tri stake difference between a unit by unit tests . the systems that store functional us . um . anybody can hopefully do we sticky on the butt . well I did it is said that some observers . so unit tests each of you Lead . . yes I'm addicted to do to leave it to all of its most hundred and eighty million Deploy suggests code . the two most popular said if you look at someone status would be to be the unit tests for a functional slash System SAS . the functional five systems thus you can run and usually with at all . a good database running already occupation actually run and something like the postal doesn't the . on delivery on the previous books API and I expect to get this back which you can run the test unless Facebook Islam is alive and well and going back to you right . unit tests . Without beginnings . so I'm saying it's those . the marina . yes code Unit tests it is a test on like a Pull told us . remarks Things and use your property unit tests were huge deal the Rhine . with literally nothing else running on the part where everything to properly mark dot com or leave familiar with mocking . . Marty and essentially were you . you . in some ways alter the code when sometimes you do change things of that . your code that you're Testing runs that is if it doesn't know any better in what's returned is actually a picture it's it's a static Things yours know what's going on in the snow Network bio or conditional . so this is something that the bell we made the decision of that simple geo . unit testing we expect High coverage functional tests and insistence at St We is a lot more at Aachen meal we do actually and staging sell . the other thing that goes on Testing CI environments ever critical somewhat a lot of years Analysis great . um the thing that I don't think a lot of people in a silly doing with with C I and one inside like to see more of In coupled with Automation what we do not support you inside . we pushed code every code that gets . is run through the unit tests we time to recommit . and then when we are done with something . while we how somebody shot off the definition of Done or a familiar with the term Definition On . . and then once that's done They Motors there Feature Blanche into what we setup the week all pending which is basically something that engineers had dictated and said In reviews as the code Complete . um there is the world in an ocean of difference between code Complete and actually Production as we all know um so happened is that it's March and a pending in an awesome treat so spending as much as a much different Use . what's ends up happening is . and then began renting Assess We and I might be included with a couple other Feature branches to make sure Feature branches are stepping on each other . I'm in an arts and will actually build it got the package can push that two out of the Tory of the latest Build . and then that is a BIG kicks out out can automatically deployed that you on the stage an environment which we then run out to Lake Build Test side as the rehab the consumption Slide System tussle Iran it's fairly automated . and though we have a BIG bread one point from Oakland and the kids actually Deploy from there . . alas they are Cycle answering documentation . he . I promise I've moved into managing engineer said . the slice of our . jockey many . what you did in more about documenting why he didn't . com . in that is I would take away . Product we go back to you and your doors one know why they're having to do something like um . I recently had Feature and I were building are in the belief are simple geo . United one idea . the BDO that the the team is a village in the setup which builds Feature to St Feature one dollars . I want builds on the task would usability many Conduct he said . we Going zeros Job . it's like we're building . and I . so what Done is actually called to live by actually document on my side . that's why we're building now . you know that the actual lightning to save back in the old is a BIG public of purity Your NMR years like that . now . the doctor that we can all look and say no in general this is what the product to do this is why redoing an arm and then the no documentation from there continues on at the mo why on Double SL . my operations guys have documentation for this couple it's not documentation around how to write a puppet and Story blue ball that it's documentation as to this is why we use CTO and two XL older We whatever else are so high CPU insisted Estate verse this thing . um it's very much predictability of March this comes in lots of code . the White becomes really important is to bring more people on because as the Lead institutional knowledge nearly all one away so it's a time without having the gospel one Team archaeological a BIG teacher your code to go wif things are are with our . . last thing I want to leave it on a time for questions and is open Cutter Chad and um and so things of about how far shore . it is true . welcome . them to lower one . Product us align same old school . I need to be User great idea to give them the million Define ways is a very anti X key thing for me to say . but um really seen the comment mean the difference you could code or viewer into comics the difference Feature good quarter in a bad CodeThe . the DiggCo railway has won what the fuck coming out the door and that could of us like dozens of them . I myself . the last five years I've always had an IV . cause two hundred code reviews . and I've never said I could are needed to find at least one show stopping by . um the oven I sing us really Continuous with purity is a particular code of years . opportunity idea there's really an achiever ways to go out this one is the news that school from which is cool bugs were basically you and I commanded you say yet I cannot looks good it looks good . one scene the fall down is going back to Carter's being lazy biggest Build what do in the don't do it . um and then also the patches are very it's not a cookie sticky Sicko they look right so I look at some ticklish leaders Bush missed can commence in earnest like one minute ten one two five one tear . really I'm a good understanding of what the Hell's going on . so we did it is made actually show a giant patch of over the whole thing as needed . in the future that latter thing when you're with is like he'll three or four five in the sitting room that a Kenyan and taking over Quote . went on to be really helpful for steals up is sad . the checklist . you know you He templates to make sure that there's no site scripting bugs and make sure that that ok the CSR Up tokens are being properly implemented . that the you know all of the standards and things they you They your spouse be doing for you . seeing that up in JavaScript on such and are in there . you know when you look at . the code are you doing anything stupid like you know running SQL queries and nine anyway actually had and things like that so it's good out on those things . in a check box in this peacefully . no you go through an agenda and was lot more quickly at the point . to editDouble wall that I wanted to chat about some any questions comments or flames . the menu . . some good . . Lewis Job . .