Even on Keynotes people don't set their fonts properly

I know it's not the aim to get Visual Studio or Blend visible, but for heaven's sake, LET ME SEE WHAT YOU DO and increase your font size. Silly!

Technorati Tags: TechEd

TechEd - Keynote

Just arrived in the room for the keynote. Of course as it usually happens, all the devices I carefully put on charge last night were not charged, which include camera and camcorder. Expect multimedia content tomorrow instead :(

The room features two screens on which bee videos are being broadcasted, as well as live video from two artists doing some real-time tagging. It all sounds very social network. The theme is orange and yellow, very cyber-opera! And for one the wireless network seems to be sustaining the load quite well!

[13:59] S. Somasegar gets on stage.

TechEd gives a chance to interact with some of you. A lot of you have had a crucial role in delivering .net 3.5, through feedback, technology previews, bugs, scenarios you highlighted and taking a bet on those technologies. For this I'd like to take the opportunity of saying thank you.

When I went for dinner on Saturday my colleagues organized, two things made it an experience: the food of course, but also the environment and how it was presented. Alike, Differentiated user experiences are what will drive the next generation of applications.

It's two years ago that we've launched Visual Studio 2005. As of today, there are 1 million professional developers using vs2005, and that's a phenomenal number. As well, we had over 17 million downloads, a mind-blogging response. At the other hand of the spectrum, we wanted to get into the lifecycle tool space. Over a fourth of our developers are using the Team System tools. But our developers are a community. A few years ago, if you asked a question you would get 30 or 40% chance of getting an answer. Today it's 80%.

Microsoft has always been since 1975 a platform company. Every piece of software is always thought about as having a platform component and a user interface component. The best thing we can do is to create a modular and extensible platform. We want to make sure we have the most vibrant developer and partner communities.

My 11yo does not know programming but spend a fair amount of time online, she's on facebook, she wants to customize her home-page, and share what she's creating. She wants to create an online experience even though she's not coding. My wife is an accountant, she doesn't code as part of her day to day work, but to automate some tasks she sometimes code. You have different experiences for different platforms for different users.

That's why we have different tools and different platforms.

As we speak, the product team is putting the final touches to .net Fx 3.5, which is the next big step for productivity. You should be able to use your .net programming when you do client, services, cross-browser web through silverlight. We already have .net 3.0 that we released in Vista with WPF, WCF, WF... AddIns in WPF, WCF protocol additions, designers being integrated in Visual Studio.

But my favorite is Language Integrated query, LINQ. You had the data world, and the programming world, and they were different things. You have to understand the data language, the constraints and understand the semantics to write your application. We said, let's reuse your skills, what if you could use the same query system as a first class citizen for your objects, for your data.

Silverlight is the new platform. We shipped the first version earlier this year. and we'll push further with the version integrating .net

While we finish the framework, we also finish Visual Studio. For the first time we're supporting multi-targeting. For the first time yu can have one toolset and build applications that target 2.0, 3.0 and 3.5. Looking at the feedback people are very happy we're going that path.

If you're building an AJAX platform, it's integrated in VS2008. If you want Vista technologies, .net 3, we have designers. It's also easier to target applications for office 2007. You used to have to buy VSTO and you get all the tools. We got feedback that it was more complicated than it needed to be, so we took the feedback and integrated VSTO in VS2008 Professional.

We have a good story for VS with server, client, office, cross-browser. We've integrated database tools as well.

And finally we released earlier this year Expression Studio to enable a smooth workflow between Expression and Visual studio.

We also want to continue delivering more prescriptive guidance. Patterns & Practices will continue and we added the S+S (Software+Services) solutions, so it's easier for you to build yoru applications.

MSDN used to be a publishing platform where we pushed articles. It was a good start but msdn should be a community platform. The community can come together through forums and be self-sufficient. We now have MSDN Wiki, took our documentation and published it and added a wiki to annotate content and add content. Just like we've translated visual studio in many more languages. Using automated translation we provide a base for content that the community can build on. Were working on the Portuguese documentation.

Currently we're working on MSDN Code Gallery. There's no easy way to share a sample code. MSDN Code gallery will let you do that. We'll release a beta sometime in the next month, and Live in the next year, so you can come and share your code samples Code gallery. I want to turn MSDN into a community platform and we're well on the way.

Today we're making available a technology preview for Microsoft Sync Framework. Take data synchronization to a new level: it's a framework to enable synchronization for your application, for peer2peer and offline capabilities. Take a look at it and give us a feedback. [EDIT: Is that the return of the sync work from WinFS?]

Software+Services Blueprints is the first guidance we release for standard scenarios. I'm a big user of Outlook 2007. My wedding anniversary is coming next month. Would it be nice if there could be an add-in that integrates the wedding date, buy on its own the present based on analyzing my previous spending, on its own without no intervention buy and deliver the present? [EDIT: What the frell?]

The Blueprint is a bunch of prescriptive guidance and building blocks

As we speak today, later this month we'll release the RTM [EDIT: I Wonder where the RC1 went? Or do they think beta 2 was good enough?], and we'll ship soon after that.

Tony Goodhew is working for the Prodct marketing team is going to do a demo of the cool stuff you can do with Visual Studio 2008.

[EDIT: I won't retranscript the demo, I doubt there's much inherent value for my readers in describing someone showing technology that's already available to people].

I hope you had a play with the beta. We've shown you what you can do with it. Let's see a video of what really happened to get Visual Studio 2008.

[EDIT: Video is slightly funny, I'll let to others the pleasure of commenting on it. Let's just say it emphasize the fact that VS2008 was built by VS2008.]

In 2005 we took a bit longer than we wanted to deliver, in hindsight, you did a good choice delivering it. On the way there was a whole bunch of tide overs, and we only managed to do what we did because we had visiblity on our project, where things needed help, more resources or scoping.

We have more than 200 partners devliering mroe than 2000 products. We're making changes to our partner programs. Some of our partners wnat to target other platforms, we're removing the restriction on using our tools to write for other platforms. We're also making the Visual Studio source code to our partners, and debug your add-in. Dan Fernandez will show you some of the products that can be done.

[EDIT: Demo of an open source project for World of Warcraft add-ons for designing worlds, it was built in a few weeks by a few developers. Show demo of LUA editing].

With Express editions we wanted to expand our audience. Popfly is a visual mashup system: as an open beta last motnh, there's been 100k customers.

With Popfly, you can publish your web page to popfly. Popfly explorer and visual studio integration will help with that.

[DEMO: Visual Web Developer and Popfly explorer beta: It's available at http://www.popfly.ms/Overview/Explorer.aspx).

Technorati Tags: TechEd, WOW

Arrived in Barcelona - The goods

Got in Barcelona alright. Had a goo time so far. Big bag of good with yet another unwearable t-shirt. Am I the only skinny person attending these conferences? An XL t-shirt when you wear XS is ludicrous.

Here's a picture with all this content. Look at the size of this tshirt, it's mental! It's probably going to join the rank of numerous ones that I use exclusively for DIY.

The bag is cool though, very good quality, and such an obvious tool to make your colleagues jealous when you go back to the office. I'll keep my Muji though, I like my bags like I like my laptops: slim.

Technorati Tags: Teched-Developers, TechEd-Developers

Getting ready for TechEd

• Configuration of Windows Home Server for remote access - checked
• Full backup of the laptop on DVD - checked
• Spare external hard drive - checked
• International access for the 3g card - checked
• 1080i camcorder - checked
• Digital camera - checked
• Lab machines assigned DNS and RDP - checked
• Online Check-in - checked

Still to do (or why i won't get much sleep)

• Finish the automatic encoding / transfer of Hi-def videos to my media server
• Finish my DV application
• Find some use for all this video equipment (maybe a daily video blog of what I saw....)
• Finish rewriting some Windows Live Writer plugins work for real-time blogging
• Finishing installing the SDKs I may need to have fun coding when there.
• Try to find out who's going and where are the good parties.

May I mention I'm thrilled? :)

Technorati Tags: TechEd

Working with Sql user instances

Just a quick note, as I've not been very active on this blog for the last few weeks. I still have a lot of content in the pipeline so expect a new tips'n'tricks article next week. I've been preparing some stuff to publish at TechEd as well, so if you are going to be around let me know I'll show you some fun stuff.

Also I'll be releasing a tool to help work with sql express user instances and attached database files that I've been working on. Hopefully you'll find this useful.

Stay tuned...

Microsoft and the Magic sweet spot

As I was coding away on the train today, for a few minutes my mind blanked while watching the landscapes Virgin offer to us regular travelers. The did an amazing job at decorating the length of the track, to the point where you'd nearly get fooled into thinking there are actually villages, cows and other things between London and Manchester. As it happens sometimes, I started thinking about my early days as a fresh and learning developer, many many years ago.

Against the normal academic training some IT people receive, I evolved as a developer from the top down. I didn't do University and got no degree in IT. I actually got as far as studying law. I learnt everything by starting to write HTML web sites back in 95, then started JavaScript in 96 and VB in 97, and discovered JScript.net and C# when I got through a friend an early preview of what was to become .net (NGWS anyone?). And I'll admit that while I understand how x86 works and how pointers work, I am absolutely unable to write a single like of C++ or any other esoteric close to the metal languages (I hope esoteric here is not taken as a challenge of C++ usefulness, as I'm sure the gurus amongst you will have understood). And it's only after all these years of coding that I start understanding calling conventions, Win32, etc.

It all got me thinking about how you understand a language and a platform. Until you reach the thorough knowledge that only years of experience will bring you, you often ignore the why question, preferring the answer to the how: you care about how you achieve a result, with the acceptance that you do not understand why it needs to be done in a specific way.

For example, in JavaScript, you'd add a method to an object type by adding it to its prototype property. You know that instance methods and type methods are two different beasts intuitively, because var $cube = new Cube(); $cube.getValue = function () { return 2; } will not work for another Cube you'd create later on. This empirical knowledge is surface magic for however long you'll take to dig deeper, learn and discover what a prototype is and how it relates to objects. At this stage, if things go wrong you'll probably have to learn more things. And it's an open ended confusing thing because both $cube.getValue and $cube.prototype.getValue can exist.

Another example comes from the WPF world.. Whenever you create a custom library of Controls for WPF, you learn about themes and how styles and templates get applied to your controls. You know that the themes being selected for a control are the end of the line customizations you provide, and you know you have to declare an attribute at the assembly level, and we're still very much in the domain of empirical knowledge. But you also know that the fallback when you don't provide themes is a file in the themes folder called general.xaml. You know the how but the why has no answer. It is a decision the framework designers took, and you have no way of changing it or influencing it in any way. This by design rule is crystal magic. The behavior cannot be changed, and once you learn how to do it, the why question is irrelevant. And if it doesn't work, there's no additional knowledge cost, because you don't need to learn anything else.

My point in all that is that these two magical elements have conflicting developer audiences. More Junior developers (and some senior developers I've worked with in the past), let's call them Mortimer, are happy with crystal magic, because it massively reduces the surface of the API you need to learn. More senior developers that have a will to understand (anyone that uses reflector as his main documentation tool would fall in this category), let's call them Einstein, tolerate surface magic because they can crack the surface, dig deeper and know all the intimate details of how the surface is achieved. They don't necessarily like crystal magic because it just works and they have no control over it, but as it's a closed ending, there's nothing that can be done about it, so they move on.

Everything would be for the best if surface magic worked for Mortimer. APIs like WCF provide a very simple surface magic, with only ServiceContract, OperationContract and DataContract concepts to learn. With the tooling support for WCF services in visual studio, the surface is small enough for most developers to start being productive very quickly. The reality strikes as soon as something unexpected happens: passing too big of an object graph, or worse, a cyclic reference within the DataContracts. Suddenly, Mortimer faces understanding notions such as bindings, behaviors, configuration files, and many other concepts that multiplies exponentially the API surface they need to understand before they can fix a problem. You just designed an API that targets Mortimer when everything works, but that requires Einstein to fix if anything goes wrong.  That said, tooling support is appropriate for being productive, which should ensure WCF adoption, so we're pretty close to the sweet point.

WPF on the other hand has a bit of both. Xaml is easy enough for Mortimer: it's understandable and can be fixed by Mortimer, because most of the bindings between DependencyObject and Xaml follows a crystal magic design. The property store will map Xaml and DependencyProperties based on the name, and nothing you want or can do will change this. You also very much pay as you go when you learn Xaml, because you can start with simple object graphs, and discover attached properties at a later time. You can use them without understanding them, and often you'll be able to fix issues by learning a bit about the Binding syntax. On the other hand, it's a very large platform and requires much more time than Mortimer would spend learning a new technology. Areas like Bindings are complex when they don't work, outputting errors to the debug window is confusing when you don't know where to look, the coexistence of logical and visual trees is difficult to grasp for winforms developers, etc.  Tooling support should help, but from what I see in beta 2, I don't believe the sweet spot will be hit with this version of Visual Studio.

And finally PowerShell is a deliciously efficient crystal magic solution. Things work as expected, pipes and formats work by magic, using a defined behavior that will not change. The framework is static enough that people are confident that the knowledge they learn using the script engine will be valid whichever Commandlet they call. The built-in variables work automatically. $_ will always have the same meaning, $$ too.

What Microsoft doesn't have anymore is a completely crystal magic solution for development. ASP was very much crystal magic, ColdFusion was too, back in the CFML days. Early days VB had few extensibility points which let developers grasp the platform more easily. PHP still has a bit of that feel, just like many other frameworks that limit choice and extensibility to increase the pace at which their developers become experts. When most of your developers know 80%+ of your API, you know you're on the right track. Anything under that and you face a problem: You're segmenting your market between Mortimer and Einstein. Your Mortimer has to get more knowledgeable to survive your technology update, your Einstein will just keep doing his expert job. The low end of the Mortimer market is gone to join the ranks of other easier and simpler technologies.

The sweet spot is not to try and provide one API for everyone, but to provide one crystal magic API for the quick and easy, and a completely different solution for Einstein and upgraded Mortimer.

Getting dynamically typed languages with the DLR is going the right direction. But now, this is a call for action to you Microsoft: Provide us with simple tools with a lot of magic to gain back the people gone to LAMP. Give us a forms design tool that maps any control on a form if the form is named after a table and a control is named after a field in that table. Give us a web tool that simply easy work with no mvc, no presenter, and a simple script language to inject a bit of server side code where it makes sense.

And remember that a crystal magic API is one that you *do not extend*. Not everyone needs a framework.

P.S.: This is just my opinion on why we're loosing so many people to php and other simpler technologies. I have a lot of designer friends that just plain refuse to touch any Microsoft technologies, because PHP is so simple. I'm probably not the target audience for any of this. Any resemblance with current or previous coworkers is purely intentional. No guarantees implied or granted. Do not use in a nuclear facility. Etc.

Technorati Tags: rant, .net, frameworks

The tale of the annoying read-only ConfigurationElement

One of the piece of code I disseminate on each project I work on is a lightweight factory builder. The idea is that you declare which interfaces are implemented by which types, or constructed by which builder. It's perfect for replacing all the types you abstract when unit testing, and put the responsibility of the dependency injection on the configuration file.

Here's a small example of how one would declare an interface as having a concrete implementation.

<factories>
  <factory interface-type="CaffeineIT.Blog.IConfigProvider, CaffeineIT.Blog.Demo"
           concrete-type="CaffeineIT.Blog.SettingsConfigProvider, CaffeineIT.Blog.Demo" />
</factories>

Very straightforward. From your code you initialize the type like so.

var myConfig = FactoryActivator.CreateInstance<IConfigProvider>();

The problem when unit testing my configuration sections is that they're read-only by default. That means I need to do some black magic swapping config files around when running my tests.

Well, it's not very well documented, but with the help of reflector, you discover the secret so many have kept to themselves. To make your ConfigurationElement read-write, override the SetReadOnly method and do nothing.

protected override void SetReadOnly()
{
    //do nothing!
}

I don't know about you but I feel like I'm cheating.

Technorati Tags: .net, .config

WPF Tips'n'Tricks #8: Use your code-behind for binding

In the View-ViewModel-DataModel pattern that has been documented a fair bit for WPF, it is often the case that you want to have a testable class that wraps your data to expose an object model that will make binding from Xaml code easier. However, when writing prototyping or simply answering messages on the Msdn forums, as well as for the code examples on my blog, it's often the case that you want to have a sort-of ViewModel that you want to quickly hack together.

The solution is to simply leverage the code-behind of your window! Let's create a simple property on our window code-behind.

    public partial class MainWindow : Window

    {

        public MainWindow()

        {

            InitializeComponent();

        }

 

 

        public bool IsFlagSet

        {

            get { return (bool)GetValue(_isFlagSetProperty); }

            set { SetValue(_isFlagSetProperty, value); }

        }

        public static readonly DependencyProperty _isFlagSetProperty =

            DependencyProperty.Register("IsFlagSet", typeof(bool), typeof(MainWindow), new FrameworkPropertyMetadata(false));

 

    }

And now in the Xaml code you just need to bind the DataContext property to the class itself.

<Window x:Class="CaffeineIT.Blog.XamlCollections.MainWindow"

  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

  DataContext="{Binding RelativeSource={RelativeSource Self}}">

    <TextBlock>The flag value is:

        <TextBlock Text="{Binding Path=IsFlagSet}"/>

    </TextBlock>

</Window>

Voila! You can now bind straight to the properties defined on your Window class! But as the complexity of your project increases, you'll want to unit test the ViewModel, and for this you'll have to separate the code in a separate class. The way I choose to do it is to embed the object directly inside the Xaml code. With this approach, you'll have the a ViewModel class. Here I implement it by inheriting from DependencyObject to keep the DP we defined earlier, as it provides support for validate and coerce callbacks, but this is absolutely not necessary. You're more than welcome to implement your property through accessors and INotifyPropertyChanged.

    public class NewWindowViewModel : DependencyObject

    {

        public bool IsFlagSet

        {

            get { return (bool)GetValue(_isFlagSetProperty); }

            set { SetValue(_isFlagSetProperty, value); }

        }

        public static readonly DependencyProperty _isFlagSetProperty =

            DependencyProperty.Register("IsFlagSet", typeof(bool), typeof(NewWindowViewModel), new PropertyMetadata(false));

    }

And now I can declare this object in my Xaml.

<Window x:Class="CaffeineIT.Blog.ViewModel.NewWindow"

  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

  xmlns:vm="clr-namespace:CaffeineIT.Blog.ViewModel" DataContext="{DynamicResource ViewModel}">

    <Window.Resources>

        <vm:NewWindowViewModel x:Key="ViewModel" />

    </Window.Resources>

    <TextBlock>The flag value is:

        <TextBlock Text="{Binding Path=IsFlagSet}"/>

    </TextBlock>

</Window>

I find the code very tidy. Through this mechanism, you can also bind properties from your view to the ViewModel. Imagine: View updates ViewModel updates View... You can abstract all the logic of your view in your ViewModel in an oh-so-clean way it makes me salivate every time. You can then leverage commands to encapsulate actions that modifies the ViewModel which update the View. No more tier leakage!

Technorati Tags: wpf, .net, .net3, .net35

.net source code to be released

From ScottGu, the source code for the .net framework libraries is going to be released, and on a source code server. This is thrilling news. Makes me regret spending all this time within WPF internals through reflector! Does that include comments in the source files?

Software that sucks #1: Why you should do your testing on the train

This is a new rant series where I moan and exhibit my frustration with using some software that has been designed or tested by monkeys. I'll try as much as I can to be constructive and propose reasonable solutions, but sometimes there's just nothing to do to save software with poor design or engineering.

One more week-end, once more another 03:50 hours train trip. I do like these travel times as I feel I knock down work for this blog the most efficiently when I'm on the train.

One thing I don't have is proper connectivity. I have a Vodafone Mobile Connect ExpressCard for my beloved MacBook Pro, which does ensure a nice and easy 3g+ connection to the Internet wherever I am (how I wish apple would include a UMTS/HSDPA chip in their laptop so I could do without the express card). But you see, while France has a vast network of high speed trains on which they ensure mobile phones work (and Paris even has phone connectivity in the tube!), Shakespeare's land is not as gifted. Reception on the London - Manchester line is dreadful, with a few areas with 3G or 3G+ (if the train is stopped), some areas with GPRS, and a few areas where the signal drops too much for anything to go through. As a result, my card disconnects and reconnects every couple of seconds. And that's where I'll do my strangest assertion to date: If you develop any kind of connected application that may be used on a laptop, plan for your testing to happen on the train.

The regular loss of connection for a short while should be and is supported by the TCP stack to a certain extent. But your application *should not* rely on the connection being there. Separating the use of the data from the reception / emission of the data should be priority number one of application developers, and timeout / crash / retry management should be baked in. Outlook does this with it's offline Exchange mode, that separates the caching and receiving of emails between local and remote store, and the displaying of the local store.

However, Internet Explorer doesn't seem to work the same way. It has trouble with displaying web sites if the TCP connection gets suddenly suspended for a while: after doing a small network trace with ethereal, the data gets received again, but for some reason wininet decides to hang in there for a while and do nothing, even if you start a new tab. Closing and reopening the browser solves the issue.

Where it makes me scream is when it's not software, but drivers that encounter these issues. Like the ExpressCard driver. Has any of the engineers working on this driver ever tried it in low reception areas? Expect crashes, hangs, your mouse blocking for a few seconds. Absolutely dreadful. And all this on Windows Vista of course.

Technorati Tags: rant, software, 3g, vodafone, expresscard

MediaElement and MediaPlayer, and the ocx control fairy tale

As some of you know I'm always monitoring the msdn forums. I usually only step in the conversation when no one else had or when it's a complex subject I know something about. That leaves me with very few contributions :)

That said, I often read from Microsoft people and from external people that anything that plays in Windows Media Player will play in the WPF video elements, that it uses the windows media player ocx under the hood, or that the content runs in WMP in another process.

So I thought I'll clear up a bit how the rendering of video is done, from the few sources I've found and from some experimenting.

First I created a small WPF application containing a small video, and I ran Process Explorer on it to see what was actually loaded.

There's a few interesting things happening there which I think are interesting to naute. First let's review the components that are running. wmp.dll is the core library for Windows Media Player; quartz.dll is DirectShow related; evr.dll is the Enhanced Video Renderer; wdmaud.drv and dsound.dll are audio related; d3d9.dll is what is used for WPF rendering and part of Direct3D; and a bunch of filters like msmpeg2vdec.dll and some .ax splitters files that don't show on that capture. Finally, we have MilCore.dll.

We know that MilCore.dll is the unmanaged part of WPF, and is responsible for composition and rendering. It's built on top of Direct3d. We can deduct that MilCore.dll is the one implementing the audio / video playing, because it exports functions such as MilMediaSetIsScrubbingEnabled, MilMediaOpen, etc.

What we do also know is that for WPF to render video inside the composited graph, it takes whatever default DirectShow graph created automagically for a specific file, and replaces the normal rendering surface by the EVR. As explained on msdn, the EVR is a new renderer shipping as part of Vista that can be used for both DirectShow and MediaFoundation pipelines for rendering and compositing video streams. It's been said before that the renderer is also shipped as part of .net 3.0. Don't be surprised if some media center applications not built on WPF still have a requirement of .net 3.0 exactly for that reason: Leverage the EVR to replace the ageing VMR9. You can find more information on these on Msdn. Finally, the EVR supports a model where you can replace Presenters, aka the surface on which the video gets rendered. And we know from an msdn post that WPF ships its own Presenter.

In passing, that's where you may encounter scrubbing issues on XP. The presenter is the only component knowing intimately about your hardware. It's as such the one that will know about the vsync on your monitor. In XP, the custom presenter synchronizes with the composition engine, but that engine is only on one window. The windows themselves are still in GDI/USER32 land. Without knowing enough of how the composition engine works, I still assume tearing still happens exactly for this reason, even though the WPF content is a retained mode scene. On windows Vista, the same composition engine is used to do a composition for a WPF application or for the whole desktop. As I understand it, it is actually only one rendering tree, where GDI apps are just a node in the tree with a graphic, whereas a WPF app has a whole visual tree, which in turn is what lets the magnifier on Vista to properly zoom on WPF content.

Well, there you are. The difference between plain old Windows Media Player and WPF is large enough (different renderer) that there's no guarantees that a video playing in WMP will play without issues on WPF. All this still begs the question as to why is the WPF team using wmp.dll rather than go straight to DirectShow? I would place my bet on not duplicating engineering work, as Windows Media Player now supports initializing either the MediaFoundation or the DirectShow pipeline based on the file, and leveraging that support would save code duplication. Or it could be something completely different.

But what we can say is that the Windows Media Player process *is not executed*. We can also say that while it's true that there's a linking to wmp.dll and the use of COM interfaces, it is not possible to say if an actual ActiveX control or OCX is used or if some code is not leveraged for DirectShow/MediaFoundation graph construction.

Technorati Tags: .net30, .net35, wpf, directshow, mediafoundation

Office Outlook Connector for Windows Live Hotmail's new version doesn't work!

Well, I was forced to do an update this morning and of course I can no longer access my Windows Live Hotmail account from outlook!

Windows Live team, with auto update comes great responsibility, and you just failed me.

Programming WPF 2nd edition

Just received this morning as part of my review of all the major WPF books, and what a shock. The first edition was quite short, the second is 800 pages!

Half way through the ChrisA, and quarter of the way in Nathan's, i'll do the Petzold last so I expect to be reading Ian & Sells book sometime in the next two weeks.

Technorati Tags: wpf, .net3, .net35

WPF Tips'n'Tricks #7: Smooth(er) scrolling

A common request with ListViews and other scrollable components is to have a smoother scrolling. By default, when a scrollable control contains items that control their own scrolling, it jumps from one item to the other, by the whole item height. It's not smooth because it's not animated, a step back in usability that I still cannot explain (lack of time maybe?)

The culprits are mainly the Panels like StackPanel and ScrollContentPresenter that don't implement smooth scrolling animations. While changing their behavior to support smooth animation is a way to solve the issue, and it may well be that I start implementing it for a future Tips'n'Tricks, but for now it's out of scope for this entry.

So how to prevent this jumpy behavior? Don't let the content decide how it's going to scroll! It's very easily accomplished, by adding the ScrollViewer.CanContentScroll attached property and setting it to false. Here's a small example:

<Window 
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Smooth Scrolling Demo" Height="300" Width="300">
    <Grid>
        <ListBox ScrollViewer.CanContentScroll="False">
  
            <ListBox.Items>
                <TextBlock Height="40">Text1</TextBlock>
                <TextBlock Height="40">Text2</TextBlock>
                <TextBlock Height="40">Text3</TextBlock>
                <TextBlock Height="40">Text4</TextBlock>
                <TextBlock Height="40">Text5</TextBlock>
                <TextBlock Height="40">Text6</TextBlock>
                <TextBlock Height="40">Text7</TextBlock>
                <TextBlock Height="40">Text8</TextBlock>
                <TextBlock Height="40">Text9</TextBlock>
                <TextBlock Height="40">Text10</TextBlock>
            </ListBox.Items>
        </ListBox>
    </Grid>
</Window>

[Update]
Ronan is absolutely correct in pointing out in my comments:

There's a drawback when using this trick: when ScrollViewer.CanContentScroll is set you loose default scrolling behaviour of the ListBox which uses a VirtualStackPanel. On big collections you will notice a great performance degradation.

This is a complete oversight in this tips'n'tricks. Virtualization means that only visible items within a list will be created and added to the visual tree. Not an issue for a few items, but when you reach the hundreds or thousands, it becomes a necessity. When you set the CanContentScroll property, the virtualizing stack panel stops virtualizing and renders everything. Be aware of that!

And as is also pointed in that comment, making a smooth scrolling *virtualizing* panel is even harder.

Technorati Tags: .net3, .net35, wpf

Multiple .config files

A good tip from Scott Hanselman. on how to manage multiple configuration files. I end up rewriting these pre-build batches every time I'm on a new project, now I'll have a central space to copy the code from!

Excel and Sql, or the joys of data massages

A problem we encounter too often is extracting data the business stores in an excel spreadsheet. The way some people deal with it is way too often to copy and paste all this in T-SQL code and have long integration scripts.

This causes two major problems. The first one is that you rely on your database to integrate data in your application from other systems. The vast majority of products have validation logic and application integrity checks into your application code, and rarely in the database schema. Not only that, but you suddenly expose as a public and exposed system what should remain an implementation detail of your service. In the end, you just made changes to your database structures an order of magnitude more difficult, which now decreases the maintainability of your code.

And don't even get me started on testability of T-SQL integration scripts, even with the new VS for database professionals unit testing. It just doesn't add up.

That said, the not to do is often the was done and conflicts with the needs to be done by tomorrow. Often enough a project cannot (or doesn't want to) afford a proper integration between their systems, as these projects come in expensive. Often, good enough is what the business wants, and the business controls the budget.

And in these cases, you often have to deal with data from Excel and Access. Few people actually know that you can query these files from within your T-SQL code, by using the OPENROWSET command, which gives you whichever table you want to select, through the table name or through a query.

In this instance I've just spent a good hour trying to understand why my excel spreadsheet opened through the OpenRecordset would return me a column full of null values when the spreadsheet itself had the correct values.

As usual, the answer is hidden in the knowledge base, KB194124. Excel tries to guess the datatype of a column, and gives you null if it fails. Rule of thumb, never trust anything that tries being smart.

Solution is to switch excel to import mode, by changing the connection string you use to add IMEX=1:

SELECT * INTO #xl FROM OPENROWSET('Microsoft.Jet.OLEDB.4.0',
'Excel 8.0;Database=file.xls;IMEX=1;', ['Festival List$'])

And I'm back in business!

[UPDATE: Added some commentary. Also changed the text around a bit, sometimes I realize my English gets fuzzy when I don't proof read before posting. Sorry chaps.]

Technorati Tags: .net, ado.net, sql

Executing Sql scripts from powershell

Realizing I still have 6 Tips'n'Tricks articles nearly ready to go to follow up on the series, but none of them are finished, I thought I'd give you another powershell trick.

One of the main issue on windows when interop has to be achieved between powershell and cmd is the issue of CreateProcess trying sometimes to be too smart about quotes. That's why powershell supports being called with a base64 encoded parameter as command line parameters, and why attempts to use variables when calling sqlcmd.exe fail miserably.

So I wrote a quick hack script that concatenates data in a nice batch file and then executes the batch file. I've used it on several projects by now. Anyone wanting to make it more fancy don't hesitate.

function Execute-SqlFile($file, [string]$Server, [string]$dbName, [hashtable]$variables, [switch]$WindowsAuthentication=$true, [string]$Username, [string]$Password) {
        $batch = (join-Path (cat env:TEMP) "exec_sql.bat")
        write-Host Connecting to $Server
    
        $output = (join-Path $env:TEMP "output_sql.txt")
        if (test-Path $batch) {
           Remove-Item $batch -force
        }
        $data = ""
        if (test-Path $batch) { Remove-Item $batch }
        $data += "sqlcmd -S $Server"
        
        if ($WindowsAuthentication) {
            $data += ' -E'
        } else {
            $data += " -U $Username -P $Password"
        }
        if ($dbname) {
            $data += " -d $dbName"
        }
        
        if ($variables -and $variables.Count -gt 0) {
            $data += ' -v '
            $isFirst = $true
            foreach($key in $variables.keys) {
            
                if (! $isFirst) { $data += ' ' } else { $isFirst=$false }
        
                $val = $variables[$key]
                $data += "$key="
                $data += "`"$val`""
                
            }
        }
        $data += " -i `"$file`""
        
        
        $data += " -o `"$output`""
        
        $data | Add-Content $batch -force
        
        cmd /c (Resolve-Path $batch)
        
        gc $output

    }

Which lets you call it like so:

PS C:\WINDOWS\> Execute-SqlScript "local.sql" "dbserver" @{key="value"; key2="value2"}

Enjoy!

Technorati Tags: powershell, msh, monad, sql

Mix:UK 07: Sneak Peeks

Bamboo flute playing with the audience.

Demo of 3D application showing someone's head captured with an MRI, followed by a foot, with bones and flesh being coloured. A sneak peek within us.

Sea Dragon: How to deal with high resolution images, that will be included in Silverlight 1.1 [EDIT: This is the technology used by PhotoSynth]. This lets you stream high resolution images. Demonstration of a commercial for a car that lets you zoom in very deeply until a corner of the image reveals text and a dashboard.

Technology to deliver ads. Project Gatineau is part of adCenter. Some of the demo should be available in beta by the end of the month. Through a report, you can see graphs based on the age of users, which is collected through LiveID.

There's also visualization technology which lets you see where people navigate. There's also a marketing campaign which shows a timeline.

There's a treemap visualization of traffic. This provide tools to let a web site owner to know where people navigate. You can subscribe for the beta on the advertisement web site.

Simon Peyton Jones from Microsoft Research does a presentation about the multicore challenge. Because of the increase in the number of cores, processors are not getting faster anymore. So you need to do task parallelism.

To do parallelism, you need to use concurrent programming objects that have been the same for 30 years, with no progress.

What's wrong with locks? First you can forget the lock, and you have a race condition. Or you take them in wrong order and you have a deadlock. Condition variables get threads to block if someone forgets to notify. And error recovery is very difficult. But theres worse. Locks are absurdly difficult. Sequential code is doable by undergraduate, locks and condition variables done right gets you a publishable paper at an international conference.

Atomic memory transaction: If we could wrap code inside an atomic clause, there would be all-or-nothing semantics: That's the Atomic from ACID. They execute in Isolation. And there's no lock.

How could it work? With Optimistic concurrency. Execute the code without taking locks. Each read and write in the code is logged to a thread-local log. Writes go to the log only, not to memory. At the end, the transaction tries to commit to memory. Commit may fail, then transaction is re-run.

How to do blocking? You could add retry. Retry says "abandon the current transaction and re-execute it from scratch". The system could keep a reference to all the reads you did and only re-exeute the transaction when one changes.

Locks don't compose well. If within an atomic code block there's a request for retry, the whole transaction retries. They also compose well with choice, which lets execute either a segment of the code or another. If code1 retries, execute code2. If code2 retries, the whole transaction will be retried.

There's a run-time cost, but a compiler technology and hardware support can reduce it a lot. It's better to have a correct program than a fast program. Finally transactions can even be faster as there's no locks and it's more fine-grained.

Graphs shows that it's actually faster than fine-grained locking, while being more scalable.

Atomic blocks will be a step forward. IT's like using a high level language instead of assembly code. It's not a silver bullet, you can still have concurrency errors. You can still introduce bugs, but they're much more interesting bugs.

There's a implementation available today in STM Haskell: http://haskell.org/ghc Theres a tutorial on STM in the book beautiful code And there's other papers at http://research.microsoft.com/users/simonpj/

Demo of a video tool that lets you drag and drop videos on a timeline in low-resolution, done in Silverlight with Sharepoint integration.

Expression Web 2 supports PHP!

LiveStation: Silverlight application streaming video from a peer2peer network. There is no browser!

There's a game coming out soon: Halo3! With actual footage from the game.

[EDIT: And finally, we've all received a copy of Vista Ultimate and a copy of Expression Studio! Which is fabulous considering I moaned about wanting a license to Expression Studio at the dinner the night before!]

Technorati Tags: mix:UK

Mix:UK 07: Cloud versus Client

Panel debate. [EDIT: You'll have to excuse me but too many names to write down.]

Chairman: First question, for Danny that comes from Google that worked on Gears. Danny: One of the first question is, what if you loose the connection? You have offline storage. Google Gears have that, Isolated Storage in Silverlight is another one. When you look at rich applications, versus rich apps on the client, you always need a backup plan for offline. For a game, where content is downloaded, you can run offline. For a big database system like email, you can emulate or proxy until you go back online.

ScottGu: The concept of offline storage is an interesting one in the web. It's usefulness is questionable, because of interop. When you build apps, you have a responsibility to make it work. It can be daunting if a new browser come, etc, if you want to guarantee availability.

Danny: I think it's a question of how in the long term we want the browser to involve.

Speaker 5: Offline issue is less a problem because silverlight streaming is highly redundant. Writing is on the wall for traditional hosting. If it's highly available, and can be delivered in the cloud, it lower my cost, why would I need traditional hosting?

Michael Ford: I commute 2h a day. I have a 3G card. I use Google reader, it's integrated with Google gear, download the feeds and as the connection goes in and out the connection is seamless. In my company we're working on a desktop application that is computationally intensive. What silverlight lets us offload the calculation engine on the client. It's also better security as it runs in a sandbox makes it interesting.

Rob Blackwell: What bothers me is the user experience piece. The evolution has been to provide richer and richer interfaces. If you look at the horse power of chips and networks, everything is available. We're nearly at the point where it's an exponential curve. We're going to see tremendous increase on the client and people will want to increase the quality on the UI, like HD DVD. When I go on holiday i easily fill 10GB.

Chairman: Angus arrived.

Danny: There's two points. The client cloud offline storage there's two model. First there's a copy of the client, or there's a cache on the client. You still need to think of it being on the cloud. As PC and devices use expands, people have access to the Internet on many different devices. Their personal life experience is available on all this, this is very empowering.

Public: We do that on email, flicker, etc.

Danny: How long after gmail did someone hacked it for a GDrive.

Public: How do you keep people confident with storing their data in the cloud?

Speaker 5: There's a lot of issues with that and it all comes down to terms of services. Service agreements are usually missing, you can contribute with terms of uses that don't guarantee you'll get your data back. Microsoft does a lot of work on that.

Michael: I have a gmail account, I archive everything, but I always have half an eye that someone is keeping it.

Rob: What about the enterprise scenario, they're not always technologically savvy.

Danny: With Oracle, if you stop paying, they take your data away. With the advent of larger data centers, on top of that there's SO. Microsoft has audits about privacy but thats just because of antitrust. There should be audits for everyone.

Public: Large amount of need for space is copyright material. If we move t the web, who controls that, if they move copyrighted dvds, music, etc.

Danny: On the thousand people uploading the same CD, there are tricks and techniques to recognize patterns. Don't know how many data centers do that. If it's cruelly high security type storage that most vendors don't do, theyre not the same bits because partitioned. There should be personal backup copies as per the law so shouldn't be an issue.

Phil Winstanley: Copyright is a big issue. Cloud based projects go and create mash ups. People can create more and share more, rather than distribute songs like they used to do.

Public: What about mesh computing and going on other machines.

Speaker 5: Silverlight streaming is about storing and delivering. I get the ability to build applications on top of other platforms that i couldn't have done on my own. It provides a model for distributing computing.

Danny: The mesh computing causes an issue if there's a single point of failure thats covered but if your connection go down you're screwed.

Public: What about WiMax? I have 250Gig hard drive on an ipod, connect it to the network. If you have enough bandwidth at home you don't need the local storage. But if the connection goes down, you dont have it.

Danny: And the more bandwidth available for you, the backbone needs to be able to scale. Not sure if the economics work.

Public: You're held hostage by people providing the bandwidth. They want a share of it for high bandwidth providers.

Dany: AT&T blames google for that situation. If google makes money out of ads on youtube then they should pay a share of the money.

Public: We're on a technological rift. We were client-server. Then the web. Now we put the execution engine back on the client, stil going to the servers. What about an execution platform on the crowd.

Michael: I prefer client applications to web applications. I don't think the browser is a fantastic platform for a rich client. With silverlight its getting better but I need my calendar always available.

Public: Why use google reader instead of outlook?

Michael: Outlook? Ouch. No there's probably clients I could use.

Speaker 5: It's different this time becase there's a huge network. Now were looking at the web as a platform. We have more functionality and services.

Danny: There's a balance that's hard to drive in either direction. There's more processor time available than a user will use at any time, whereas how many technical problems fit into a mesh network. It doesn't work on little probems very well.

Phil: Browsers are not designed to do what we make them to do. We can't wait for the browser to give the functionality anymore. Thats a fact of development.

ScottGu: The browser has a lot of challenges. From the distributed perspective, the browser changed some things for the better. Each app used to be a silo talking to one database. Now you can link to a blog, to a profile, my mom using an app doesn't know using the application goes to the web. Silverlight, flash or server processing. The web taught us asynchronous programming in a distributed way. The expectation is for the app not to freeze. Considering how wide is the network we work on. You gonna see people use AJAX, client, Silverlight, Flash, and combine them. The end result will be richer than it's been on either the server or the client.

Danny: Cross domain security for the browser. The browser is secured for cross domain. At the time it was a reasonable compromise, vertical and monolithyc, except viruses. But vendors have evolved faster the browser and we now hit the issue.

Speaker 4: If you use the conchango application on the USB stick, its going online for some services, but it works offline. If you have a client running many operating systems then you want the browser. If you control the platform, you can do client. With Silverlight you have an easy way to leverage your assets.

[EDIT: Missed a question there]

Public: What can the panel say about issues with the cloud?

Danny: Issues with the cloud, there were vulnerabilities, then they could get a huge amount of data in a very short time. That's why datacenters have proactive security, and kill parts of the network as needed.

Angus: There's a bot that's more powerful than the fastest supercomputer in the world. That's waht happens. Security: first version of facebook. Viral invites of people, the installation of the application exponentially.

Speaker 1: What happens with privacy, there's been a couple of password. Did the password come back or is it a new password. If it comes back, it's been stored unprotected. What if someone hacks in the system, suddenly they have all the passwords.

Danny: On the same vein, to do our Live Services available to developers, we have to do these exercises, the user has to see the data transferred. That puts the load on the user and makes the process more difficult.

Angus: It's at the detriment at the user experience. Nothing can be rosy when you share a bed around.

Public: TKMaxx had issue, it's not only the cloud, it's the same issue.

Public: Originally, we did online bank access. They now send card readers to swap your card and PIN [EDIT: SmartCard]. But on the other hand they send faster. If i'm in the USA I can't. The very big commerce sites will start doing propriotary ways.

Public: The net will end up in anarchy and end the human race.

Danny: I'd like to comment on the behavior changes. We see that already. The young generation have a completely different view of their personal data. Things like myspace, they don't have much worry about who see their data. There's already a cultural shift.

Phil: People are more open, they write a public diary 16-24, and what they put in there is quite, hum, interesting. Personal knowledge of someone is very powerful.

Public: Cambridge people were showing themselves on pictures getting drunk and they got fired.

Rob: My father used to distrust banks, and keep the money under the mattress. There's been a change there.

ScottGu: Same issue existed 10 years ago. Now they just take this for granted. If a company wants to stay running they have to protect my information. Today there's a lot of issues with law firms, can you buy people's private data. That's going to be an interesting shift.

Angus: Some company wants to spend money to know me to tell me what I want.

Public: From a disaster recovery point of view, with the cloud everything is still available. If you need high availability you're better off on the cloud. Access is not an issue.

Danny: In your home network you have the same reliability issues. You share documents but if a machine goes offline or your wifi goes offline you can't edit your word document

Angus: I think replication across the cloud is the killer application.

Public: Is there a way to have server only code?

ScottGu: First, can we have a core framework, with core set of libraries? That's what we use in silverlight, Server Core in Windows Server Core with no GUI, with IIS. You can't run .net code yet but we'll see in the future. You'll see more factored versions. Silverlight is the first version where we pulled all these things out.

Public: There's so much data now compared to a few phone numbers before. Young persons now just drop their email address when there's too much junk. Data becomes very disposable. As we go on how do we deal with that volume of data ad use it to our advantage.

Danny: I think someone said that information available in our day to day life per person is equivalent to one you had at all 10 years ago.

Public: 8 or 9 years ago, it was LiveJournal, then myspace, facebook. But they change, trends change. Wikipedia may be available for a few years.

Public: How do we get around legal issues across boundaries, data stored in the US about users in Europe for example. How does that work with a cloud.

Danny I do know that companies specialize in offshore data.

Phil Winstanley: mp3.com has shown that it was legal in their countries. It's unresolved.

Angus: if you shift a lot of data to China, its going to be expensive, you're better off having a datacenter there.

Danny: As for corporate use, privacy and protection, security metrics will be about IP addresses and MAC addresses access. As part of SLAs and Terms of Use, there will be clauses about where the data have to be stored. It's building fences that data centers traditionally don't have.

Michael: its no more of a problem if a small company starts trading in another country. Like antifascist laws in Germany prevents you from providing Nazi content.

Danny: Can a local market censor for everyone what one country sees.

[UPDATE: Added names for speakers and corrected some spelling mistakes]

Technorati Tags: Mix:UK, Mix:UK07

So far so good...

Lot of typing but its worth it. Hopefully people will find the content useful. Note that I mostly retranscript what's being said, so when I type I its not me talking. My comments are in bracket with a big EDIT: in the front :)