Moving on to greener pastures

After a very successful run of technological achievements on my current project, it seems there's human incompatibilities that can't be patched and I am to suffer a cold reboot. As one of my previous clients wants me to work on some of their projects, I'll suffer only minor downtime. I can't say the same for this project I've not even been asked to do a handover for, but I wish all the best success to the team I've worked with, as I know that, with the IT people, everything ran smoothly and we did well. They will do well. Farewell.

In the meantime I'm taking a week off, so I'll be able to post the content I've had pending for a while, including MVP, SQL user instances and more WPF stuff.

Managing managers follow-up

I just wanted to point people to Joel's Evidence Based Scheduling. A lot of people often misinterpret my rant about estimates being an impossible dark art as going against tracking like the one developed by FogCreek (damn I wish I was attending their talk in London). The reality is that what I'm saying is reflected in the model:

The mythical perfect estimator, who exists only in your imagination, always gets every estimate exactly right.

Most estimators get the scale wrong but the relative estimates right.

And the one I'm the most in tune with.

When the project begins, the technical managers go off, meet with the business people, and come up with a list of features they think would take about three months, but which would really take twelve. When you think of writing code without thinking about all the steps you have to take, it always seems like it will take n time, when in reality it will probably take more like 4n time. When you do a real schedule, you add up all the tasks and realize that the project is going to take much longer than originally thought. The business people are unhappy.

Inept managers try to address this by figuring out how to get people to work faster. This is not very realistic. You might be able to hire more people, but they need to get up to speed and will probably be working at 50% efficiency for several months (and dragging down the efficiency of the people who have to mentor them).

You might be able to get 10% more raw code out of people temporarily at the cost of having them burn out 100% in a year. Not a big gain, and it’s a bit like eating your seed corn. Of course, when you overwork people, debugging time doubles and a late project becomes later. Splendid karma.

This all comes down to estimating a release date, certainly not thinking that your original estimate will be any accurate, because it won't. What do you end up doing?

If you do make a schedule, even before you start working, you’ll realize that you have to cut something, so you’ll cut the easy/fun feature and just do the useful/important feature. By forcing yourself to chose some features to cut, you wind up making a more powerful, better product with a better mix of good features that ships sooner.

I'm feeling happier today. Now go and read Joel, I don't agree with his prose all the time but his articles are deep and show a level of understanding of the writing software process that never ceases to amaze me.

WPF ClearType and the case of the blurry font

While at TechEd I had lunch with IanG and the discussion around the blurry text in WPF came around. I mentionned that this was due to sub-pixel positioning: fonts are now positioned based on where the font designer intended to put it rather than at the pixel boundary. You gain in accuracy but you loose in sharpness. I also mentioned that fonts were rendered fatter on mac to account for this blurriness, and spent a couple of days wondering if I said somehting very stupid.

So I googled a bit and found where I got some of these ideas. Let me direct you to Joel on Software for the beginning of your cyber journey towards font understanding. I am lead to believe by all I've read that at small font sizes ClearType doesn't respect the font weight and spacing properly in GDI ClearType, while it's respected much better on the WPF ClearType, making it much closer to MacOS X sub-pixel rendering.

The problem, as Joel points out, is that users don't like change. They end up considering WPF text as blurry, when a mac user would find them absolutely appropriate. Should Microsoft backtrack on their ClearType WPF work? The problem would be that to respect the pixel grid, you need a pixel grid, and in WPF you don't have one. Ouch.

One thing I forgot to mention as well is that some of your blurry text comes from using Menus and other windows with transparency. This will disable ClearType altogether and switch to gray-scale rendering. When you wonder which mode each window is, make a screen capture and zoom, you'll see the red and green artifacts. And that includes every menu you use in your application (how's that for a hidden gem).

[Update]

While browsing wikipedia I stumbled on the following text.

Increasingly, new approaches to screen rendering have reduced the importance of extensive TrueType hinting. Apple's rendering approach on Mac OS X ignores almost all the hints in a TrueType font, while Microsoft's ClearType ignores many hints, and works best with "lightly hinted" fonts.

One additional potential explanation of the difference between GDI and quartz sub-pixel rendering.

Technorati Tags: wpf, cleartype

Managing managers

I've worked on many projects in my career, and just like my style of development has changed, so have my managers. This is not a tutorial on how to manage your manager, as I don't believe you should. No, this is more of an open letter to those managers that seem to enjoy making your life hard when it could be so simple.

I've been advocating Agile methodologies for years, I've seen big design first fail miserably and tdd and xp succeed fantastically. I've also seen the reverse, with very well organized big design first projects, often on complicated enterprise-scale systems, with many developers split into smaller teams that then adopt whichever methodology they are comfortable with.

But the worst cause of failure has always been this strange beast that is the methodology-agnostic manager. By manager I mean anything from Project Manager to Dev lead to self-appointed boss, and by methodology-agnostic I mean someone not following a proven way to ensure you can actually deliver software. This can range from Scrum (that I'm covering here) to various waterfalls to ones grown in house and prooved by time. I'll call him Ralph for the purpose of this rant. And Ralph usually say any of the following.

We'll just drop the unit tests.

This usually comes with We'll document [remember, the unit tests are your main source of documentation!] and the very satisfactory I planned a handover phase on the day you finish your contract. It usually come with a justification: We can fix the bugs after the release date, so we'll be on target.

You just cornered yourself into certain failure, Ralph. This may do for v1, you may survive it for v2 with a lot of time and a lot more money, but by v3 you've failed. Let me repeat that: You will not survive beyond v2 if you do not bet on quality over quantity.

Just do it

That one is a bit borderline. There is always pressure to deliver software, and cutting corners is a fact of life for any software project. After all, shipping is choosing. However (you've seen that one coming didn't you?) this means cutting functionality to ensure quality.

Ralph has a tendency to be terrified at the idea of managing the customer. A new project starts, and sometimes the marketing guy over-promised. Sometimes, the customer has described what they want, and we've provided a rough estimate at the beginning of the project. Sometimes, the business asked, got told yes and now Ralph does not want to go back saying it's not possible to deliver.

The reality is that if you can't deliver, you won't. Trying to hide that fact is ridiculous, you'll end up cutting features or being late anyway. It's better to catch these things early, involve the customer in deciding where to re-prioritise. Otherwise Ralph will make the choices for the business, and probably ask his development team to work over-hours. At that point you've failed.

The other reality is that predicting how long software will take is a dark art that is bound to be inaccurate. I'd probably say that as the complexity of a project increases, the probability of reaching an initial estimate drops to 1 in 5 for any reasonably complex scenarios. You can be a bit off or a lot off, but you won't get what you initially planned.

Ralph loves his excel and this does not fit with his model, and that's understandable. I know the complexity of the tasks I'm doing right now, got a few ideas about the ones in the future, but I have no way to predict how it will all work in 6 months. And I've never seen a single project that, 6 months on (or even 3 months on), still has the same requirements, the same deliverables planned and the same functionality in. Business requirements change and they often get shaped as the software evolves.

What do you do then? The same as you did, you give a rough estimate on how long to deliver the core functionality you describe in use-cases or user stories. Then you shape your project on each sprint to split between the minimum needed to deliver that functionality and the refactoring / framework-ization you deem necessary.

When hard choices are to be made, you'll often find they are about which bugs to fix or which refactoring or new code you want, rather than actual user stories being fulfilled. And that's manageable.

There's a school of thoughts in XP that advocates delivering the minimum amount of functionality and refactoring until you get a good enough solution. However, this is only applicable to applications with good code coverage (> 80%). Parts of your application built without TDD or without tests are probably impossible to put under tests easily, and refactorings become increasingly dangerous and expensive. In those instances you're probably better off rewriting from the ground-up. Remember, your prototype is *not* your v1.

This won't get used for long

Ah this one is by far my favourite of all. The first rule is that the prototype is your v1. It *will* end up being used, and will take ages to be decommissioned, if it ever is. Your v1 will try and make this usable, but you'll be stuck with the original design. The customer will not understand why your v1 cost so much more for a few improvements over the prototype. Ralph will not have the courage to say that the prototype didn't cut the mustard.

The only way to sooth the process is to hammer to Ralph: The prototype is not your v1. I know it's in prod and it works but it is *not* a v1. You can ensure it doesn't get used further by putting time bombs, erasing all the data every day and changing where your app is deployed on every release.

Just quick fix damn it!

This is also a common pitfall. It is perfectly reasonable in some instances to walk around an issue you cannot resolve. You document it and discuss it after investigating the problem and finding the root cause.

This is where Ralph comes in. Usually you describe in your daily that an issue has arisen and you need to investigate. Ralph sees a risk, and immediately will start pressuring for any investigative work to be stopped and a fix to be found. Under some circumstances it's reasonable, under most implementing a work-around for an unknown root cause will just remove the symptom without knowing the illness. You just left for later a big giant loophole in your code-base. It is sometimes the right thing to do but rarely.

Scrum doesn't work

This one is fascinating. Scrum is a fabulous methodology, but just like it is very difficult not to stop doing test-driven development under pressure, it seems terribly difficult for the newly appointed scrum master not to fall back into bad habits: micro-management and authority.

All these things happen because Ralph feels he is the one giving orders and splitting the tasks, and as such needs to keep control. This does not work. Let me repeat that. This does not work. A scrum master is there to relieve any impediments on writing software. The product owner decides what gets priority. The development team is self-organizing, because the development team is the one building the software. The devs build it, the scrum master doesn't. Ralph doesn't. So Ralph shouldn't decide, and that's a good thing.

It is good to loose control over the minor details. I'll offer you two reasons.

First, only the developers understand the complexity of writing code for the project on the platform they target. If Ralph has a development background, it was probably years ago and chances are, because he's a project manager now he's not writing any code. He can't decide on how things get done because he doesn't understand them. Then Ralph spends an awful amount of time and effort trying to understand the details of each bit of functionality to regain control. This is the worse scenario as any development planning, meeting and decision has to involve him. Ralph is now needy and has become an impediment.

Second, Ralph will work more efficiently with his colleagues if he's not constantly fighting for power. The product owner decides where the priorities are and which direction the dev team will adopt. Remember, the product owner is the voice of the customer. It's whoever decides what gets in the software. Ralph doesn't use the software, he's not the user. He doesn't talk to the users, that's the product owner. As such he has no idea what bits of functionality the customer wants to implement first. And he has nearly no control over developers because he's not the one building the software and is not involved in writing the code. If Ralph continues to try bossing people around about the code, it will end up impeding the dev team's capability to deliver.

Of course sometimes you end up with the absolute worst case scenario where the scrum master and the product owner titles are both given to Ralph. That's a recipe for disaster.

The daily stand up

This is way too common. Ralph has other issues to discuss, ranging from adding coloured cells to his excel spreadsheet to being explained a component's functionality. So you end up with No we'll do this meeting first and then we'll do the stand-up. You can see the face of developers when they realize they're going to sit around a table and watch a discussion that has very low impact on them they didn't intend on participating in: after all they came for a daily stand-up. Or you end up with we don't need to stand-up for stand-up meetings, a corollary where the purpose of the stand-up is not understood so the format is not respected.

So here's my rule of thumbs to know when you're screwing up: If you hear a developer moaning before the meeting because he knows it's going to end up taking ages; if you sit down; if the discussion is between two people and the others watch: you just ruined your stand-up meeting and the rhythm that's supposed to be associated with it. You got everybody bored and lost an hour of their time.

One last thing to any scrum master out there. You're supposed to track down how many tasks get completed, how many new ones get added, and with those numbers do your estimations on the likelihood of completing your sprint. If you don't do that, the nice cards on the board probably don't help anyone. Analyzing those tasks is what will let Ralph do his spreadsheet, if he's not following them he's not doing his job properly.

The planning meeting

Well this one is easy, it takes time, we write down all the new tasks, we update the product backlog, decide what we want to achieve for the next iteration based on the prioritisation of tasks agreed with the product owner.

However, old habits die hard, and it is overwhelmingly difficult for Ralph to accept that the planning is there to plan the current sprint, not to answer the question of how many months the whole thing is going to take. It gives a good idea of where you are and how likely you'll match your target, but it is in no way there to let Ralph write his excel spreadsheet for hours and hours. It's Ralph's job to understand what is going on and report to the business, it is not the developers'. Ralph, you're the one in charge of excel, embrace the process and keep the planning you sell to the business to yourself.

The post-mortem

I like calling it the post-mortem but there are many other names for it, from wrap-up to release party to the more common sprint retrospective. This is the moment where you analyze why you've not matched your targets, what you've done well and how you can improve the process.

This goes bad when the blame game starts being played. The only way to get something efficient out of a post-mortem is for everybody to understand and agree that estimates are exactly that, estimates. This also means that you accept that you're late and can explain what were the reasons, ranging from your lack of skills to dependencies on other components. No one should get penalized for encountering problems.

The half scrum

Another common misconception is that you can adopt a few bits of Scrum and be successful. I've never seen a project with a bit of scrum in there succeed. Neither have I seen a big plan project succeed without a very strong structure to compensate for the unreliable nature of software development. All in all, because Ralph can't decide which bits of a methodology he's going to use, he uses a bit of everything. This is a recipe for disaster and confusion.

The split responsabilites

This one is tricky. One of the most important bits is the work split. Having each developer work on a different component is a normal occurrence. However, you'll often find that on small teams, it is more efficient to work altogether on one thing rather than split each developer on each project. Assigning ownership to part of the projects to one-man team destroys communication between developers, reduces the rate at which they can code, and ends up more often than not with an inability to deliver anything in a sprint: You do half of two tasks instead of finishing one task well.

Ralph often likes to separate his eggs and have each developer in charge of one bit, so he can go and manage each of them individually. It becomes much easier for his excel spreadsheet. This silo development undermines the concept of working as a team, and often comes from Ralph trying to regain control on a project he has no visibility on. As we've seen before, visibility is there, but Ralph refuses to change his ways and embrace the reason behind agile. This is once again a recipe for disaster.

Conclusion

Here's a recap of all the bad habits Ralph has:

• He gives up code quality to try and achieve deliveries
• He's taking away from the business the choice of what gets delivered and the prioritisation of those elements
• He lives in fear of telling the business how things are really going
• He believes it will be easier to deliver now and refactor later (the prototype is your v1)
• He does not care about finding the cause of an issue and focuses only on the work-around
• He can and need to understand and decide how the software is written and who writes what
• He has to give orders (it's often the case that Ralph is the project manager but not your line manager...)
• He doesn't respect the daily meeting
• He wants to fill his 12 month plan and doesn't track the sprint backlog (the tasks on the whiteboard)
• He wants the help of developers to do his 12-month plan, because the tasks are not enough (after all he doesn't track them)
• He believes the plan is the solution to everything
• He believes he's good enough to reinvent a methodology by mixing bits and pieces
• He believes in splitting people to split tasks and regain control (silo development)

This Ralph is a disastrous scenario and very rarely will you encounter people showing all those signs. But if you try to deliver software in this day and age, and you can see in your project management more than a few of those points, I would advise extreme caution, as I believe you'll end up failing on your deliveries and ending up with a unmotivated team.

On a bright note, Ian Cooper told me once something that I kept at heart: everybody tries to do the right thing. Not knowing and fearing are normal things, failing is part of learning, and people willing to evolve overcome those limitations. Great managers are the ones that take the knowledge from their coworkers, embrace that change and adapt to and work with and for the people they manage.

TechEd: Reflection, code generation etc.

I'm sitting in Roy Osherove's session on lightweight code generation. Roy is a formidable speaker, but there's probably a bit too much content and some areas are, I believe, not explained in enough depth to be approachable by neophytes.

Overall, a very good session with a lot of deep content, and a great overview of what is new in .net 2.0. However a lot of the focus is on solving real problems through these technologies, and a good dose of DLR and expression trees discussion would probably have been a great and smooth transition to the rest of what we've seen this whole week.

And Roy is singing I don't want to blog about it. What an extraordinary end to a very good talk!

Technorati Tags: TechEd-Developers, TechEd

TechEd will be over very soon...

I'm not taking the plane until tomorrow, but it's very sad to see everyone starting to leave. The exhibition center is closed. Only one talk remain where I'll go and have a chat with Roy Osherove.

What to do tonight? I've not managed to learn about any parties, maybe tonight will be the first night where I can see Barcelona. Or maybe not.

To be fair, I am tired, my brain is close to exhaustion with all the talks, arguments and chatting that's been going through, and I want to go back to my partner and have a relaxing weekend. Which means suffering the plane tomorrow morning, an experience I'm pretty sure I won't enjoy: I have a ticket for the evening but *need* to arrive in the middle of the afternoon in London.

Oh the fun!

Technorati Tags: TechEd-Developers, TechEd

TechEd: SeaDragon and PhotoSynth

This is a private session I ended up going to because of my craving for orange juice..

PhotoSynth is in Silverlight, but it's not clear which version it will be shipped with. It's a multi scale system, the size of objects is not important. The maximum info needed at any moment is whatever is on the screen. It supports pictures, text or even a fractal.

Another demo is an application of the scaling with an issue of the guardian, zooming out to show al pages and zoom in into a corner to see a very close text that you normally would need a microscope to see on a real printed animation.

This works with the hardware of the macine because the GPU is used. [EDIT: I thought silverlight was only software, or is there something more in the pieline?]

Another demo is a bunch of photos from flicker of Notre Dames, and generate a 3d reconstruction from that picture, and where all the cameras were positioned. You can then do a photo to photo navigation.

It's all about large navigations of photos and 3d, and SeaDragon became Photosynth. The system starts from images, extract features, then those feature get matched, and the matching turns into a 3d scene.

There's some work that ha been done in the summer with NASA. A picture of the shuttle display recrangles that are features: scale, orientation and corners.

Descriptors are informative about what the content is, but minimally informative about resolution, point of view, cropping, etc. If we extract the same features for a number of images, we manage to match them. All the features get in a feature index, look for nearest neighbors, and that gives matches between several pictures. Now that you  know that two points on two pictures match and you do that across pictures, you can find out where the camera would have been and where the point would have been.Each point in the 3d visualization is one of these match features.

The system used to take several days, and we're getting ready to release in the spring community photosynth which will let a single user capture a 3d environment on his own. You start being able to capture a real part of the world. What it will allow is for users to make their own synths and share them, and link them together. When you link it you can navigate seamlessly from one synth to the next. We're going to see overlapping synths.

We've also been exploring public collections.

One of the secrets in photosynth is that it's almost 3d. That Synth was done yesterday after 5 mnutes of crawling. Some of these transitions are less than ideal ones. If you notice, the reason is because all these photos are projected on flat planes. It doesn't correspond to the geometry of that place. It's in between 2d and 3d. Something that a lot of peole have asked us to do real 3d reconstruction. There's been a flurry of research around it, and some of our colleagues have done is fabulous. There's an experimental viewer made oer the summer, with a project with the BBC. It's reconstructed 3d, we're not projecting on flat planes anymore we're projecting onto 3d models reconstructed from these pictures.

Another demo used in Live Labs is very close to the birds eye morphine in virtual earth. In virtual earth there are models of the city, photogrammetry, gps, etc. In this case we don't use any models, they come from the photos.There are still some errors but we're not finished with

We're working on integration with Virtual Earth because it relates to the interplay between top down and bottom up representations.

Technorati Tags: TechEd, TechEd-Developers

TechEd: The irresistible Forces and the Moveable Objects

Pat is a fantastic speaker, as always. However as always, the content can be followed by slides as well as by listening. For some reason, all the rules about not reading your slides and not putting too much content on them, he doesn't follow. And by some speaker magic, he succeeds brilliantly. Goes to show that, to each their own style, as long as you captivate the audience!

The topics remind me a lot of what I've heard from many previous conferences, Microsoft or not: Just imagine when everything you know will have changed so much you won't be able to apply any of what you know now. The big topics are storage, massive parallelisation of work, the death of the hard drive, etc.

What is less ground-breaking is one of the demoed video. It looks terribly similar to a digital home video that was available with the second edition of Bill Gate's book. Interactive TV, interactive Phone and interactive Car. Except where you used to hear windows you now hearthe software + services. Oh dear!

To be fair, the fact that these videos are vision of where Microsoft wants to go means that while marketing memes and corporate memos come and go, the company has kept one way forward and keep on pushing it forward.

But the most important thing to get from the talk is the idea that consistency is not as important as you think. It's a subject a lot of people from Amazon, eBay and other companies that scale massively have learnt and are now starting to promote. It's less important to be right than it is to be consistent.

In the end, I would have to wonder if a talk that apply to a handful of companies worldwide was really what should've been the general highlight of the last day.

Technorati Tags: TechEd-Developers, TechEd

TechEd: What's new in PowerShell V2

[EDIT: I'm thrilled by this presentation, can't wait to see the new bits!]

Jeffrey P. Snover [EDIT: Posted on this on his blog]

Before we start talking about v2, lets' talk about v1. The response has been phenomenal. We had a discussion with the Marketing guys. I explained all these things that happened in the past and it'll be a long time before we reach a million download. Was I wrong! We had more than 1.5 Million downloads in the first year.

We have tools shipping today using PowerShell, exchange, MOM, etc. We have more than 20 partners producing powershell inside. We're now a CEC 2009, which means any microsoft server software has to be  powershell compatible.

Why the success? You can see the tasks for exchange with VBScript, the productivity for powershell is phenomenal. A customer had a 481 lines of script he replaced with one line thanks to WMI and html formatting.

The bad: Remoting is Cmdlet specific. There's no uniformity you can rely upon. Theres a great scripting language but you can't write Cmdlets in PowerShell. Our scripting environment developement is notepad and ps-debug, which is a pretty lame experience. Finally our API does not address common GUI needs. The normal model is admin tool built on top of powershell. The people doing that end up rewriting the same thing so we are doing it.

The big things for v2 are: Universal code execution model, GUI over PowerShell, Production Scripting, and community feedback.

So what is universal code execution model? Any code that runs should be able to run on one or more machines, LAN or WAN, restricted or not restricted environments, foreground and background, in short or long connections, using impersonation, and initiated by user input or by events.

The code execution model: Invoke-Expression works with computer names, the runspace, the throttle, the credentials, the ShellName, and connection information.

If you want to run it in a backgroun, you can use start-psjob, and the parameters are the same

When we talk about remoting, the scenarios we nail are fan-out 1 to many. The goal is to reduce the TCO. Second is Interactive, and i don't want to use telnet. We're not trying to replace telnet. We're not doing the graphics telnet. Next, Many to 1. If you write a service, you dont want to create a new process for each user, that would blow up. I want to listen on a port with WS-Management. Then we want the GUI over CLI.

Mobile object model. I want to run some code over there, I bring the objects back here but they're exchange objects and we don't have the dll. The way we do that is standardizing the model, we take the type, we serialize in a property bag. It's a stable cross-machine cross-time. Get an object, serialize. It takes one of the 22 types, if it's not there you call ToString. You can specify which serialization you want. Those objects we serialize everytime we reach a RunSpace boundary.

As soon as something crosses, we serialize, and when we reach the destination runspace, we deserialize. We store the original type and we store where it came from. I have a bunch of machines, i do a getFreeDiskSpace and you get the objects back.

[EDIT: Demo of using Invoke-expression].

Background jobs: we had an object called psjob, you can provide a runspace to it, or not. If you don't provide everything we'll run it locally. We require the latest version of WS-Management (1.1), this is included in Vista SP1 but there's no download, so you need SP1. We think the background jobs should be in process, but in this CTP it goes through the remoting.
Jobs can contain child jobs that can contian child jobs.

Restricted Runspaces are environents you can host to run potentially hostile requests safely. You get control of which language features are available, which commands can be executed, and there's white listing of commands.

There's a whole new set of hosting APIs. YOu can only issue one request per run-space. RunSpace pooling, which Exchange did, is a pool of runspaces and execute the job against that pool.

There are a new set of hosting APIs that bypass the parser completely. You can provide parameters, commands, to build a pipeline. There's also a typed invoke, and redo invoke.

[EDIT: Demo of the new hosting API.]

Production scripting: We have a grpahical PowerShell, Script Cmdlets, Script Internationalization/Data Language- You go to the web and solve the problem by downloading a script. It's 3 lines of code. Now imagine it's 3000 lines, it's gonna take a while before you put it into production. Well PowerShell can guarantee it's only data. You look at the lines of code that are not data to evaluate if you run. In scripts you want to highlight which parts of a script you don't need to delve into; Debugging; and finally packages and modules.

Graphical PowerShell is done in WPF it seems.

Script Cmdlets supports most everything a .net Cmdlet can do. You add attribute. You also have the $Cmdlet variable that you get in .net as your base class.

The engine does a ton of work for you. You change from the fucntion name to the Cmdlet. You can add a Mandatory attribute, a Position, an Alias, ValidatePattern etc.

The data language is a limited subset of languages, with comparison operatirs, if / elseif / else, well defined convertfrom cmdlets, limited variables, comments and pipelines, literals.

Data msgs { convertfrom-stringdata 'commands' }

then you import the localized data with Import-LocalizedData msgs.

We have a new command called PSBreakPoint. They can be set on lines, line number / column number, functions, variable read/write, or a breakpoint any time this command is called.By default we hit a breakpoint and break. Alternatively, you can provide an Action scriptblock to execute when you hit the breakpoint. We also have get-PSCallStack. It's a placeholder, but it will be great. In V2 it will be an array of stack information, with location, scope etc.

We're making progress in including community feedback, but we are working on the big bits first.

We now have a splatting operator. We also added a -split and a -join. We added a type for ADSISearcher.

 

Technorati Tags: TechEd-Developers, TechEd

No more content for today...

Sadly I've not found conference rooms with power plugs, so no more content for today. I'll probably do a quick update tonight of what was said but it may be fairly short.

Off to see IanG for his WPF talk.

TechEd: Networking in Windows Vista and Windows Server 2008t

.Why do we need to discuss the new network stack? We want faster applications, we want to connect to anyone else, and we want a simpler, direct model without yet another server. For all these reasons we need the next generation TCP/IP.

There are some issues with Vista network issue. The Multimedia Scheduler service.  Mark Russinovitch published an article about it [EDIT: Here's the link to the article about the interaction with networking and multimedia scheduler]. One hopes this will be fixed by SP1.\

The new tcp/ip stack is a huge project for Microsoft becase it's the first-time microsoft re-write since the early 90's. It was an outdated technology. NG TCP/IP ships with Vista and a slightly different in Server 2008.

The major improvements include receive window auto-tuning, Compound TCP for very fast LANs, ECN - a way for routers to let the client knows when it's overloaded to reroute the data, better support for lossy network - RFC 2582, 2883, 3517, 4138, and some great IPv6 network with Neighbour Unreachability Detection.

There are other benefits too:  much simpler API, security enhancements with API filtering and monitoring, support for stack offload, and multiprocessor scalability, which was not possible before because of NDIS 5.1 limitations.

There are also a few conveniences: there's no restart, there's auto-configuration and self-tuning of IPv4, and policy-based QoS. There's for users roaming support in IPv4 and (better) in IPv6, home network support has been simplified, and there's more efficient multicasting.

So far, it may change in the future, but so far we have full resistance to all TCP/IP level DoS attacks, and IPv6 for more security. You also have the concept of routing compartments.

Let me try and make you a bit enthusiastic about IPv6. We are running out of address. Don't believe me, believe these numbers. ARIN announced on 21th May 2007 that we will run out in 2010. RIPE 55 Meeting took place two weeks ago said we have 2-4 years before running out, and Mr Vint Cert said that on BBC news too.

Theoretically, we could live without ip addresses. We could have several levels of NATs but it blocks us from performing p2p networks. NAT makes peer-to-peer very difficult, because of security. We need to find a balance. Seucirty of IPv4 doesn't exist. IPSec is optional, not so in IPv6. IPv6 provides significantly better QoS than IPv4. Routing tables will shrink with IPv6. Mobility does not work in IPv4. Finally device autoconf doesn't work so well. Some technologies has been backported, but it doesn't work very well.

6 benefits of IPv6: Address depletion solved, ent2end p2p communication restored, mobile and roaming connectivity, international mis-allocation, autoconf.

Terminology, we still talk about Hosts, we have LAN segments, Links, Subnets... In IPv4 a subnet is restricted to a few devices within the same network. Any enterprise network has a wardrobe of devices to solve the subnet. You don't have subnets communicating with other subnets communicating thanks to a Cisco router.

We have 128 bits for our addresses. We usually split it with 64 bits for subnet IDs, and 64 bits for interface ID.

IPv6 is expressed as 8 blocks of hexadecimal 16bits components.

A great thing for IPv6 is that there are classes instead of subnet masks: The CIDR notation.

There are 3 classes of address: Unicast, Multicast, Anycast. There are no more broadcast address. IPv4 wasn't designed for networks with very large amount of machines within the same submask. Now it's gone, you'll get a performance boost. Instad of broadcasting, there's a bunch of neighbor networking protocols.

There's a new unspecified address, and there's a new Loopback: ::1. There's the concept of a well known address. DNS Servers are supposed to be FEC0::FFF::1 FEC0::FFFF::2 and FFEC0::FFFF::3. As a developer, be aware that you should be able to accept v6 addresses.

Configuration can use DHCP, where the address is generated. What they don't need to perform anymore is the address generation. The other thing is that addresses expire. That gives us back some efficeincy and clears the address space.

Addresses expire, and one of the key things in IPv6 is that you have multiple address. You have global addresses, one expired, one in the future, some local addresses have local addresses, or several link local addresses. It's the key change: in IPv4, when designed, all the processing was on routers. In IPv6 we change the balance, nodes have enough power to do some networking process. That lets us have autonomous networks, by moving.

Mobility is the coolest thing, it's so simple and powerful it's beautiful. When a device tries to connect, it can't get any neighbors. Then it creates a local address. If the router contributes, you can change global addresses but your home address will make sure data keeps on flowing.

VIsta has what is known as a Dual-Layer architecture, isntead of a Dual-Stack implementation,w hich meant that the ipv6 TCP and UDP was grouped together and completely separate from ipv4.

Interoperability is probably a key issue for anything being introduced. IPv6 was built-in for interoperability: ISATAP, 6to4, 6over4, Teredo, portProxy. Teredo is very important because it bootstraps the world from IPv6 to IPv4. However it is not without problems.

That brings us to peer-to-peer networking. P2P goal is to enable direct communication between applications without relying on centralized  servers.  You could keep commnciating with everyone by keeping connections opened. You need to prevent DoS, Secure with crypto, resolve changin addresses, maintain dynamically changing group of nodes, and communicate with a subnet.

The p2p in vista is based in IPv6. It's installed by default in Windows Vista. At heart you have low-level and high level APIs.

The concept of a cloud is a group of peers that can communciate with each others. There's a global cloud with everybody, and a link local cloud, or several private global clouds.

A peer is a machine runnnign the application that connects to other. It has a Name and a PNRP ID, and a Certified Peer Address.

[EDIT: This is overall a big presentation about clouds, chrod based networks, graphs, etc. I worked for too many years with gnutella / gpulp / jxta to contiue blogging about the presentation. Battery is nearly dead anyway.]

Technorati Tags: TechEd-Developers, TechEd

TechEd: Threat Modeling

As an industry, we're good at looking at code compared to looking at security. You can run all the tools you want, you'll find security bug but you won't uncover any design bugs: they're the hardest to fix.

At the very high level, threat modeling is about identifying threats, mitigate them and move on. A couple of years ago, the threat modeling information we were giving was pretty lousy, you couldn't use it much if you were not a security expert. That's what we've been focusing on over the last few years.

When the Windows Mobile team did their first Threat Model, they had to answer a question: Is a stolen mobile devide still in the threat model. If the bad guy has the device, are you still trying to mitigate? They decided to try and give a fight, and include it.

For Windows Server 2003, we've asked the team to think about an admin browsing on a Domain Controller. If you compromise a DC, you can compromise the whole environment. The question always come to mobile code, and we decided to deactivate all mobile code.

Finally, you need to define who your users are. If your user is an IT guy rather than a mother, they're different people. When you present a mitigation to a user, will they understand it?

You have to model your application. We use Data Flow Diagrams for that. You model data, which is where most of the security issues come from these days.

Most whiteboard architectures are like DFD. [EDIT: Won't reproduce the diagrams, the process is defined on Larry's blog.]

When DFD talk about processes, they're not .exe, they're whatever process data.

You need to define in your DFD trust boundaries. You need to decide if two processes, when they talk together, if they trust each others. In DFD it's explicit.

For Vista we decided to add this functionality to run some processes with lower privileges, like IE runs in protected mode, and there's a trust boundary between IE and the OS. A process boundary may be between the system process and the user, as you don't trust data coming from the user. Finally Kernel transition is also a trust boundary.

Theres several types of DFDs. Context diagrams give a high-level content. In level 0, you dig into more details. I've rarely seen much deeper than Level 1. If you go over, you're probably in analysis paralysis.

One feature that never made into Vista was the Castle project: it was where workgroups meet the domains. In a DC the domain authenticate you. In the home, you have several computers but no domain. If you have 5 computers and no DC, how do you synchronize files, passwords etc across machines. So we came with the idea of a castle, a machine that was responsible for synchronizing the passwords.

Local user ---> Castle service -/-/-> Remote castle

The level 0 DFD contains many services. The shell runs as the user and communicate with the castle service: There's a trust boundary.

The castle service runs as SYSTEM because it has to manipulate the SAM. Config data in the registry is accessed by the Castle service but there's no trust boundary. And finally the machine boundary.

Purists would say that a service should be a complex service and be described in more level, but if it all runs within the same process boundary, having a simple process is good enough.

Each DFD is susceptible to certain kind of threats: STRIDE. [EDIT: again, Larry has the description. Saves me some typing]

Example of repudiation. My brother in law lied when sending a check. He said it was sent, I thought he was lying. I got the proof when the check arrived, because they were all post-marked. You use a disinterested and trusted third party.

Each element in the DFD is subject to attack. How you attack them depends on the element type. You attack data stores differently than you do with processes. How do you protect these elements?

You need to determine threats. You have primary threats and secondary threats.

If you have an external entity (a user for example), you're subject to spoofing. It's subject to repudiation. Data flow and data store can all be tampered. The vast majority of DOS come from processes, rarely against data stores. Finally processes are subject to all of the STRIDE attacks.

Secondary threats come from threat trees. They've been around for a long time. They're a bunch of pre-conditions that can lead to a threat. Unless you're a security expert, you do not know what the pre-conditions are. We made away with threat trees, and work on canonical threats.

For every primary threat, we build the threat tree. At the top of the root you have the prime threat. For example spoofing a user, there are 4 ways of doing it. For the first way, you have several reasons, and several causes which lead you to the leaf nodes in the tree. You can ask very interesting questions about the leaf nodes. [EDIT: Example about user spoofing].

The example of analyzing a bank log-in system. The connection is SSL, the password has to be strong. However the cookie was predictable, as it was i+1 for each new authentication. Suddenly by predicting the cookie you can hijack someone else's session.

We have in the SDL book all the threat trees and all the questions you can ask about the tree nodes.

Every single information disclosure threat can become a privacy issue. If you have an PII you can get a lawsuit. Any data store can be attacked.

Now you need to calculate the risk of every of these threats. We used to use numbers to calculate risks. but they're always wrong. Plus, the more degrees of freedom you have in the calculation, you can easily fudge the numbers so you don't have to fix the bugs. The more numbers, the easiest.

We used to use DREAD. I don't like it. Even David posted a weblog about it.

Calculating risk with heuristics with a simple rules of thumb: we remind you of that with the MSRC bulletin rankings. The problem with numbers is that if the number is under a level, subjectivity will permit you not to fix it, whereas the MSRC you have no way of doing that. You ask is it remote / local, deos it crash the machine, is it a server or a client product. The different levels are all described in the book.

Now you need to mitigate the threats. You have a bunch of options: leave as-is, it's low priority, trying to fix it may delay the product etc; remove the product like we did with castle; remety with technoogy counter measure; warning users, but i don't believe in the prompt, you push the security of the system on the user instead of taking ownership of the problem, its not a good solution if it's your only mitigation.

Each STRIDE threat can be mitigated. Spoofing, you have authentication. The authentication technologies you use are different: Amazon.com uses SSL. They use encryption, authentication and integrity. Tampering, you can have integrity: Digital signatures, hashes, etc. Repudiation threat is mitigated with non-repudiation, strong third parties etc. Information disclosure: You have confidentiality, like ACLs. DOS, you have availability. Elevate of Privilege you have authorization.

[EDIT" Won't take notes on the castle example, so i can enjoy it. Maybe Michael could publish that on his blog as an example!]

Technorati Tags: TechEd, TechEd-Developers

Skipping the first session

Even though I went to bed early, still woke up exhausted, so I'll give a miss to the first session. Such a beautiful day, it's a crime to not have some of those sessions outside.

Technorati Tags: TechEd

TechEd reactions

Here's a few things that went under my radar in the blogosphere about teched:

• Gill gives an overview of his first day, and think Barcelona is dirty at first sight
• Joao talks about the Influencer Community Camps pre-conference. He also talks about the CSD influencers dinner on monday, but I'm afraid I was not invited, neither was I on tuesday for the MVP (as you know, I'm not an MVP :)). But tomorrow I'll attend the MS Influencers Party, Wednesday the country party, and Thursday I'm meeting up with some people from the Windows Live team. So much to do, so little time!
• Sascha talks about the World of Warcraft IDE microsoft will release with their partner as an open source project based on Visual Studio shell that was demo'd during the keynote. Go and check it out. And Ronan let us know he'll be in the Ask The Experts for Team System.
• Andrei talks about the Microsoft Sync announcement, personally the one area I'm the most thrilled about, so thrilled the SDK is already installed and I'm coding away :)

Overall, not much noise so far. Maybe the PDC next year will be louder! More content to follow, and tomorrow pictures and videos!

Technorati Tags: TechEd, TechEd-Developers

TechEd: Security Development Lifecycle: Development Practices

[EDIT: By Michael Howard, definitely a legend in my book!]

Until 8 years ago, i was completely focused on security features. But a small group of us, around 1999, decided to split from the Windows group and focus on how to design software.

Security people suck. They only see things in black and white, you can either have secure or not secure. The real life doesn't work like that. Theres plenty of shades of gray, and security people don't see gray. What they're good at is to tell you when youre wrong, but they won't tell you how to do things.

We added the NX to WindowsXP SP2. The first wave of AMD devices supported the option of not executing data, as most security vulnerabilities get introduced as data that then get executed.

When something is not on by default, people don't turn it on. Also, if it's not on by default it won't get as much testing.

When we activated NX by default, Flash, Acrobat, Java VM don't work. So we had to choose between not running by default and not get used, or be on by default and nothing works. As a security guy I said break the applications. But my manager said that didn't work! So we decided to have NX on by default only for opt-in processes.

The one thing I learnt is that there are shades of gray.

References to books for Writing secure code for Windows Vista and Writing secure code.

So what is the SDL? It's a bunch of requirements and recommendations, and the post release process. It's taking the lifecycle and improve the security by reducing vulnerabilities and reducing the severity of the vulnerabilities.

Lot of people think you can improve the quality by adopting some process, you'll improve the security of the software: there's no evidence! You can't think of general quality stuff, you need to focus on security itself.

The matrix I use is the amount of bulletins post-shipping. Since Vista, we have fixed 70 vulnerabilities in XP, 33 in Vista. In a perfect world it would be 0, but I'm happy with the 50% reduction. IIS6 only had 3 bulletins, sql server had one 4 years ago. Even the analysts start recognizing this.

The SDL is made of two big chucks: Requirements and recommendations (must and should).

Some functionality was invented a long time ago and was fine 20 years ago. But today it doesn't work because of threats. For example CRT (C runtime). You can't write new code that uses the old APIs, 120 functions. You can include banned.h which works with gcc and others, or use VS2005 that puts a warning.

In the SDL, if you put forward a requirement, you need to be prescriptive about what the fix is. For the CRT, we recommend two libraries: Safe CRT and strsafe. VS also automatically replace the banned functions if it knows the destination buffer size.

One of my favourite tool is SAL, Standard Annotation Language. It helps static analysis tools. The shell team decided to annotate the shell headers. He ran the tools we have at microsoft again and found 100 bugs. It lets you annotate your header files with how attributes relate to each others, if they're optional, in, out, etc...

Integer arithmetic issues can be resolved in several ways. First, use the latest C++ compiler, it protects you against the operator::new overflows. It crashes your process but that's better than running code injection vulnerabilities. Second, review your memory allocation: Is there any calculation to calculate a buffer size? Does the bad guy controls one or more operations? Finally, you can use IntSafe (C) and SafeInt (C++).

Next thing is the no weak crypto. A lot of algorythm that were ok a few years ago are not ok anymore. No new code can use MD4 and MD5 (collision of 1 in a million in MD5), SHA1 is showing signs of weakness, so it's banned too. Use SHA2. Can't use DES or triple DES - use AES. RC4 is not allowed because it's a stream cipher, no symetric keys < 128bits, no RSA keys, no weak random number generation, no embedded secrets and be crypto agile: don't define which crypto you use in your code. You should also not have weak ACLs: we found a bug a couple of years ago where a vendor was intalling their device driver in ProgramFiles. They changed the ACL to Everyone full control. Michael's advice: don't touch ACLs. The defaults are very good, don't change them.

We also have requirements about compiler switches. The GS flag adds random numbers to the stack: you need to guess the cookie we've set because we check it, and if it's not what we expect we crash the process. When the blaster worm happened, win2000 and XP were infected. Windows Server 2003 wasn't infected but the coding was there. We detected the buffer overflow at runtime and crashed. It was affected but not infected. For some arguments we copy the data earlier in the stack. In Vista, we compile with the orcas compiler.

Safe exception handler would have prevented CodeRed. The PE header has the address of the exception handler. At runtime the OS checks the exception against the list in the PE header. NX is the No Execute, ad DynamicBase is base address randomization for OS code. Finally, /Strict MIDL generates safer IDLs.

Although tools are incredibly useful, they don't make the code secure. Bad code and bad developers won't be saved by tools. There shoudl be nothing special about security, each developer should include it as part of getting the job done.

In Vista there's an option to detect any heap corruption and kill you on the spot.

The only scenario in which reliability guys have a conflict with security guys is when we bluescreen the machine if the security event log is full.

There are also web-specific guidelines. A lot of people write a lot of rubbish code. Use the Xss library, don't let SQL injection happen. Deny access to the underlying tables, only allow views and stored procedures.

Gadgets are Html / javascript / Silverlight. If your gadget renders untrusted data, you can call anything. Any exploit code can run. The first thing you should do is validate all your incoming data, and code review for insecure use of innerHtml, document.write, and Eval (which is one letter away from evil and there's a reason for that).

Inspect your Gadget from Michale Howard & David Ross.

Technorati Tags: TechEd, TechEd-Developers

TechEd: Metropolis - Interchangeability of Operations

[EDIT: Unbelievable but you have to sneak in the conference rooms to get access to the one power point that's hidden under traps. Hopefully no one will ask me to unplug myself because of Health and Safety but you never know... An IT conference without power plugs, what a great idea!]

I've used the term Metropolis for a set of talks. Look at Cities, Transportation, manufacturing and try and find some trends. From looking at cities we can learn a lot about what will happen in IT.

At a high level, IT shops map to cities. You have evolution that occurs in isolation, and then things get connected, which causes change. Factories and buildings are comparable to applications. Each application is part of an IT shop. Retail maps to business process. It's about tying together deliveries that come from different origins.

Manufacturing goods and structured data and operations are similar. That what we see as data structures and operations are made to fit better together.

I don't believe it's an analogy, it's not helping to think about a subject, it's the same drive behind both technologies.

What are the challenges in SOA? Explicit boundaries; Autonomy - you can replace your service without impacting the other services.

How do you perform actions across these boundaries. What trust do you use? SOAs are behaving the way you see people in business behave in the real world. Its an opening up of the autonomy and explicit boundaries.

Those that know of me know I've been doing transactions for a long time; but I've grown to believe that transactions across boundaries are very fragile. I don't think services as boundary of trusts will engage into transactions with other services. Some say two-phase transactions will happen across services, some say they shouldn't because of trust issue. For this talk, each service will have a transaction but will not hold locks for another service.

When you look at manufacturing, there was a transition from hand-crafted to automated. A machine creates the part. The shape and form was so crude with the machine that you had to manually change them before assembling. Interchangeability was driven by being able to exchange pieces between two broken guns on a battlefield to make one that work.

In computing we're massively stuck because we don't do very well on how to exchange services. The important points that you need to take from this transaction:

• Autonomy
• Agreement is different: Attempt and confirm/cancel
• Interchangeability: One operation is as good as the next. Fullfill my requirement, be it that you use A or B. You execute your operation on a pool of services.
• Semantics: How do you make the request to rearrange and reorder. You need Precision but you don't want the Intricacy.
• Variety: It can be made out of interchangeable of the operations. If you put different goods in WalMart, the combination is astonishing, but they come from a limited number of pools.

Let's see the history of manufacturing. If you go back to the early nineteenth century, you'd order one thing and you'll get one gun. You have a lot of craftsmen, each of them builds a complete item. But each person made a different gun because it was based on their own approach. Completing anything needed fitting. Soft parts were stamped because they couldn't be exchanged.

The American way of manufacturing comes from shortage of labour. They started building machines to do the work. The problem was that the machine created parts were still inaccurate. Because of that, they need adjustment. Fitting was still required.

With the armory system of manufacturing. LeBlanc proposed interchangeable small parts but it was dropped, but it was brought back in America, or at least the idea. By creating Jugs, Gauges, and the precision work John Hall, the price of manufacturing the pieces increased because you had to be picky with the pieces you get. You had interchangeability but the economics didn't work.

From armories to bicycles to sewing machines, all the way to cars with Ford, impacts what we do today. The disassembly plant: the meat arrived in Chicago, and each butcher was responsible for cutting one piece of meat. There was very high throughput. In assembly, Ford moved the parts to the people rather than the people to the parts.

The Ford factory made sure there was no fitting, every part had to be fully designed, custom machines were made, so all the pieces could be interchangeable. The machine was giving the precision and the accuracy. Even the building was shaped to optimize the production. But it became impossible to change the car design as you would now have to knock the building down.

Interchangeability is great but there are other requirements! Which brings us to General Motors. GM gradually introduced the concept of a yearly model, which introduced the requirement of being agile. You suddenly need multi-purpose machine tools. For this you had to be able to re-arrange the components of the machine to create a new piece. Mechanization, interchangeability and rapid changeover to new models.

Let's talk about the American system of transaction processing. You start your activities, be it a human or a machine, and it goes to your database, you calculate ad you return your answer. But you have to do fitting inside the transaction. There's no boundaries when you update two databases within a transaction. Within each transaction you need to fit the work. But it gives you volume.

We need transactional "machine tools". No precision tools: our current tools can be used in different ways. Different programmers get different results. They're wonderful for labour savings, but not to separate their behaviour. Today, we have applications that have external behavior that are similar but the inside is different. If we are to take our applications apart, we need to make the internals of our applications interchangeable.

Services are connected by messaging. You don't know how the operation is executed, but you know the contract of the messages and the contract of the order in which messages are exchanged. Interaction is based on business functionality. The services share operations, not data. Sometimes there's reference data. It's like a department store catalog, you use that data to connect to your business.

What about optimistic concurrency control? Can you do optimistic concurrency work with your bank? No, because there's a security boundary. You need to think about trust.

Autonomy means independent control. My local business logic decides how i change my local data. I decide what changes. If you want a change, ask me to do a business operation.

Long-running work. Services don't share transactions. How do they cooperatively make decisions? how do independent businesses take decisions? There's tentative operations, with reservations, cancelable orders, or confirmation. If you buy a house in the states, as a buyer you give everything to the Escrow company. The bank, the buyer, the seller, everyone puts their trust in the escrow company. However the sell is only a reservation until you know if you will get it. The only guarantee is that you'll get your money back if the deal doesn't work out.

Coordinating n-systems requires that at least n-1 accept uncertainty. You system needs to be in a confused state. In 2-phase commit, the database maintains locks. With cancelable operations, uncertainty is the reservation you have for a hotel room. There's an intrinsic conflict between consistency and availability. Two Phase commit is the anti-availability protocol.

Because operations are cancelable, you can reorder them. Your cancellation is not an undo but an action you receive to cope with the failure. So what are the semantics for cancellation / confirmation?

Cancellation is about coping with the operation not being done. You accept the right to cancel. Confirmation is the guy that has the right to cancel saying that he won't, for example hotel room confirmations in the morning. Airline companies clear up their overbooking once the plain leave.

To make your operation cancelable, you want to reserve its effect until the confirmation. If an operation is unique, you must lock its effect until you cancel it. You need to make a decision between provisioning and overbooking. How do you manage the pool of resources?

One operation in a class is the same as another. If you reserve a king-size room, it could be any of the rooms. Interchangeability comes by increasing commonality.

It's a pain to offer cancelable operations. But you want to do it because your customers need that in order to function correctly. Its annoying that the customer can be fickle but it's the reality of a complex application.

We used to make decisions atomically by committing or rolling back your transactions. You want to now think about how things compose, but for that you need to have each service being part of a long running work.

In the armory system of transaction processing, you have a complete solution when you assemble small services. In manufacturing you need precision. You need to understand the constraints of your operation. We've made components that were multi-purpose. What you need to do is to have the least amount of functionality to make them interchangeable.

Resource-oriented data lives longer but is changed by long-running operations, like bank records. You have activity-oriented data, that gets created when you start doing a job, and retires when the operation is complete, like an order.

These two classes of data have different characteristics.

In both cases, the data is encapsulated within the service, and there's no optimistic concurrency control externally. You mediate access through business logic.

Let's take a Foo operation. First case, the data is isolated. If I change something about someone's order, I only change a bit of data. But for Joe's order, that may modify my resource-oriented data by changing my inventory. Activity oriented data is used for a single long-running activity and won't span several. You can create a tentative operation, it's easy to track down in case the operation is canceled, but nothing is shared so there's no impact. What happens if you share data?

It becomes harder in resource-oriented services. What is interacting with what? They make commitments and these commitments need to be tracked. You have an association between activity data, the reservation, and the resource oriented data, the list of hotel rooms that are available every day.

Specificity is not interchangeability, Variety is the enemy of interchangeability.

There's a lot of work around DSLs. You express the intent of the business domain with a domain language. That's a great foundation to think about the issues that have been raised.

This is heavy-duty reconfigureable manufacturing machines. You don't need much skills to use DSLs if they're constrained enough.

Service operations are like standard parts, whereas tentative operations compose. That lets you do standard software components and use them across several products. All the products look different on the outside but they're all the same inside.

We should learn about that manufacturing part, and do some rambling philosophy.

Atomic transactions are singularities. New challenges happen as you spread work across space and time. Interchangeability helps relax space and time. You can interleave and reorder. Variety means you have lots of options and choices, but you can't keep stocks. Interchangeability needs precision rather than variety.

EDIT: Some form of question about Wal-Mart using Child labour and how it relates to services. Sometimes reality is better than science fiction!

Update 26th November 2007: http://blogs.msdn.com/pathelland/archive/2007/11/25/presentation-of-metropolis-interchangeability-of-operations-at-teched-emea-in-barcelona.aspx

Technorati Tags: TechEd, TechEd-Developers

Even on Keynotes people don't set their fonts properly

I know it's not the aim to get Visual Studio or Blend visible, but for heaven's sake, LET ME SEE WHAT YOU DO and increase your font size. Silly!

Technorati Tags: TechEd

TechEd - Keynote

Just arrived in the room for the keynote. Of course as it usually happens, all the devices I carefully put on charge last night were not charged, which include camera and camcorder. Expect multimedia content tomorrow instead :(

The room features two screens on which bee videos are being broadcasted, as well as live video from two artists doing some real-time tagging. It all sounds very social network. The theme is orange and yellow, very cyber-opera! And for one the wireless network seems to be sustaining the load quite well!

[13:59] S. Somasegar gets on stage.

TechEd gives a chance to interact with some of you. A lot of you have had a crucial role in delivering .net 3.5, through feedback, technology previews, bugs, scenarios you highlighted and taking a bet on those technologies. For this I'd like to take the opportunity of saying thank you.

When I went for dinner on Saturday my colleagues organized, two things made it an experience: the food of course, but also the environment and how it was presented. Alike, Differentiated user experiences are what will drive the next generation of applications.

It's two years ago that we've launched Visual Studio 2005. As of today, there are 1 million professional developers using vs2005, and that's a phenomenal number. As well, we had over 17 million downloads, a mind-blogging response. At the other hand of the spectrum, we wanted to get into the lifecycle tool space. Over a fourth of our developers are using the Team System tools. But our developers are a community. A few years ago, if you asked a question you would get 30 or 40% chance of getting an answer. Today it's 80%.

Microsoft has always been since 1975 a platform company. Every piece of software is always thought about as having a platform component and a user interface component. The best thing we can do is to create a modular and extensible platform. We want to make sure we have the most vibrant developer and partner communities.

My 11yo does not know programming but spend a fair amount of time online, she's on facebook, she wants to customize her home-page, and share what she's creating. She wants to create an online experience even though she's not coding. My wife is an accountant, she doesn't code as part of her day to day work, but to automate some tasks she sometimes code. You have different experiences for different platforms for different users.

That's why we have different tools and different platforms.

As we speak, the product team is putting the final touches to .net Fx 3.5, which is the next big step for productivity. You should be able to use your .net programming when you do client, services, cross-browser web through silverlight. We already have .net 3.0 that we released in Vista with WPF, WCF, WF... AddIns in WPF, WCF protocol additions, designers being integrated in Visual Studio.

But my favorite is Language Integrated query, LINQ. You had the data world, and the programming world, and they were different things. You have to understand the data language, the constraints and understand the semantics to write your application. We said, let's reuse your skills, what if you could use the same query system as a first class citizen for your objects, for your data.

Silverlight is the new platform. We shipped the first version earlier this year. and we'll push further with the version integrating .net

While we finish the framework, we also finish Visual Studio. For the first time we're supporting multi-targeting. For the first time yu can have one toolset and build applications that target 2.0, 3.0 and 3.5. Looking at the feedback people are very happy we're going that path.

If you're building an AJAX platform, it's integrated in VS2008. If you want Vista technologies, .net 3, we have designers. It's also easier to target applications for office 2007. You used to have to buy VSTO and you get all the tools. We got feedback that it was more complicated than it needed to be, so we took the feedback and integrated VSTO in VS2008 Professional.

We have a good story for VS with server, client, office, cross-browser. We've integrated database tools as well.

And finally we released earlier this year Expression Studio to enable a smooth workflow between Expression and Visual studio.

We also want to continue delivering more prescriptive guidance. Patterns & Practices will continue and we added the S+S (Software+Services) solutions, so it's easier for you to build yoru applications.

MSDN used to be a publishing platform where we pushed articles. It was a good start but msdn should be a community platform. The community can come together through forums and be self-sufficient. We now have MSDN Wiki, took our documentation and published it and added a wiki to annotate content and add content. Just like we've translated visual studio in many more languages. Using automated translation we provide a base for content that the community can build on. Were working on the Portuguese documentation.

Currently we're working on MSDN Code Gallery. There's no easy way to share a sample code. MSDN Code gallery will let you do that. We'll release a beta sometime in the next month, and Live in the next year, so you can come and share your code samples Code gallery. I want to turn MSDN into a community platform and we're well on the way.

Today we're making available a technology preview for Microsoft Sync Framework. Take data synchronization to a new level: it's a framework to enable synchronization for your application, for peer2peer and offline capabilities. Take a look at it and give us a feedback. [EDIT: Is that the return of the sync work from WinFS?]

Software+Services Blueprints is the first guidance we release for standard scenarios. I'm a big user of Outlook 2007. My wedding anniversary is coming next month. Would it be nice if there could be an add-in that integrates the wedding date, buy on its own the present based on analyzing my previous spending, on its own without no intervention buy and deliver the present? [EDIT: What the frell?]

The Blueprint is a bunch of prescriptive guidance and building blocks

As we speak today, later this month we'll release the RTM [EDIT: I Wonder where the RC1 went? Or do they think beta 2 was good enough?], and we'll ship soon after that.

Tony Goodhew is working for the Prodct marketing team is going to do a demo of the cool stuff you can do with Visual Studio 2008.

[EDIT: I won't retranscript the demo, I doubt there's much inherent value for my readers in describing someone showing technology that's already available to people].

I hope you had a play with the beta. We've shown you what you can do with it. Let's see a video of what really happened to get Visual Studio 2008.

[EDIT: Video is slightly funny, I'll let to others the pleasure of commenting on it. Let's just say it emphasize the fact that VS2008 was built by VS2008.]

In 2005 we took a bit longer than we wanted to deliver, in hindsight, you did a good choice delivering it. On the way there was a whole bunch of tide overs, and we only managed to do what we did because we had visiblity on our project, where things needed help, more resources or scoping.

We have more than 200 partners devliering mroe than 2000 products. We're making changes to our partner programs. Some of our partners wnat to target other platforms, we're removing the restriction on using our tools to write for other platforms. We're also making the Visual Studio source code to our partners, and debug your add-in. Dan Fernandez will show you some of the products that can be done.

[EDIT: Demo of an open source project for World of Warcraft add-ons for designing worlds, it was built in a few weeks by a few developers. Show demo of LUA editing].

With Express editions we wanted to expand our audience. Popfly is a visual mashup system: as an open beta last motnh, there's been 100k customers.

With Popfly, you can publish your web page to popfly. Popfly explorer and visual studio integration will help with that.

[DEMO: Visual Web Developer and Popfly explorer beta: It's available at http://www.popfly.ms/Overview/Explorer.aspx).

Technorati Tags: TechEd, WOW

Arrived in Barcelona - The goods

Got in Barcelona alright. Had a goo time so far. Big bag of good with yet another unwearable t-shirt. Am I the only skinny person attending these conferences? An XL t-shirt when you wear XS is ludicrous.

Here's a picture with all this content. Look at the size of this tshirt, it's mental! It's probably going to join the rank of numerous ones that I use exclusively for DIY.

The bag is cool though, very good quality, and such an obvious tool to make your colleagues jealous when you go back to the office. I'll keep my Muji though, I like my bags like I like my laptops: slim.

Technorati Tags: Teched-Developers, TechEd-Developers

Getting ready for TechEd

• Configuration of Windows Home Server for remote access - checked
• Full backup of the laptop on DVD - checked
• Spare external hard drive - checked
• International access for the 3g card - checked
• 1080i camcorder - checked
• Digital camera - checked
• Lab machines assigned DNS and RDP - checked
• Online Check-in - checked

Still to do (or why i won't get much sleep)

• Finish the automatic encoding / transfer of Hi-def videos to my media server
• Finish my DV application
• Find some use for all this video equipment (maybe a daily video blog of what I saw....)
• Finish rewriting some Windows Live Writer plugins work for real-time blogging
• Finishing installing the SDKs I may need to have fun coding when there.
• Try to find out who's going and where are the good parties.

May I mention I'm thrilled? :)

Technorati Tags: TechEd