Saturday, 24 July 2010

An intro to what OpenWrap is

I vowed not to come out too early with loud propaganda on what OpenWrap is, but many have told me that they want to see it happen now rather than together with a beta, and I’ve been coding away for long enough that the code may be of interest to some. So, without further do, I announce my latest venture in world OSS domination called OpenWrap.

What’s OpenWrap

OpenWrap is currently classified as a package management system. In reality, it’s an export management system. The reason for OpenWrap existing is to deliver stuff to whatever environment is needing it, be it at development time within Visual Studio, at deployment time when pushing an OpenRasta or MVC web-site, or at run-time for a rich client.

What’s Stuff?

An application is usually made of a bunch of stuff. Some are components, some are static files, some are executable tools, and some may be an energy pattern that needs to go from point A to point B.

OpenWrap is a simple packaging format that deals with separating those stuff things into exports. An export is whatever you want, and only the consumer of an export need to know and understand a certain kind of export.

An example of exports: assembly references and build files

One of the many exports that everyone is focusing on at the moment is binary assembly distribution. You want to easily add a component built by someone else, without the hassle of having to download binaries and all their dependency hell yourself. I hear you! While projects like horn have attempted to take away some of the pain by automating cross-trunk version building, the lack of binary distribution and versioning has made using the tool a hit-or-miss, because of the nature of how open-source software is built more than because of what horn is.

So, in OpenWrap, you can use the OpenWrap shell, called o.exe, to issue a bunch of commands. So if you want a reference to nhibernate, you’d do

o add-wrap nhibernate

OpenWrap will try and find the package called nhiberante, and all the packages it depends on, and install them in your system repository. At build time, each package you depend on will automatically get its references added to your project. There’s even a ReSharper plug-in to make it real-time.

In your package, which happens to be a simple zip file, each dll present in the /bin-netXXX folder will be automatically resolved. Why the XXX? Because in OpenWrap, the binary exports are dependent on the version of .net you’re running under. That means that if you run under .net 2.0, you’ll get all the DLLs present in /bin-net20, but if you’re under .net 3.5, you’ll get the ones in /bin-net35.

Want to provide architecture-dependent code? Just put the same dll and the same API in both /bin-x86-net20 and /bin-x64-net20, and OpenWrap will figure out which of the versions to use.

Another example of exports: commands

In OpenWrap, commands are also a type of export, that live in /commands. Want to provide command-line commands to execute as part of your project? As long as your package is present in a solution, all of the commands you create will be available from the command-line. XCopy command-line extensibility is the feature used by openwrap itself to execute its own commands, so you have the guarantee that only the version of OpenWrap (and other packages) you have in your project repository will be the ones executed when you are running the shell.

The sky is the limit

Of course, package exports are extensible, so if you want for your project a custom export, or if you feel (like me) that web applications should share a same export for things like jQuery, then you can integrate yourself with that and make it your own.

Repositories

There’s a few repositories in OpenWrap. The System repository is the one living in your /Users/UserName/AppData/Local/OpenWrap/wraps folder. Anytime you execute an OpenWrap command anywhere that’s not an OpenWrap folder, it will come from your system repository.

As you may have guessed by now, each of your projects will also have its own repository. You can consider that an equivalent to your vendors folder in Rails. In each of your projects, all exports (commands, assemblies etc) come from your project repository first.

And of course, multiple repositories can exist. The repository at http://wraps.openwrap.org is maintained by myself, and you should be able to upload your own packages there very soon. If you want to add your own repository, the openwrap-server will provide you with a full-fledged repository you can install locally. You then only need to issue a o add-remote myRepository http://myRepository.com command to add your own.

Versioning

Whenever you execute the o add-wrap xxx command in a project, a line will be added to your descriptor file. This file is the description of what your application does, what other packages it depends on and what versions it needs to execute. The end-result in your file would add a line. Here’s a typical package descriptor, to give you a sense of what things look like. (note the syntax is slightly different than what is on the wiki, those changes are coming up on github in the next few days as we ramp up for release).

openbong.wrapdesc

Depends: nhibernate >= 2.0.0 and < 3.0 Depends: openfilesystem = 1.0.0 Description: A wiki system based on the textile language and using GIT as a repository.

Versions are very simple, you decide what your package is compatible for. When it comes to dependencies, only [Major[.Minor[.Build]]] are taken into account. If you add the fourth number, the revision, openwrap will always update the package to the latest revision.

Of course, you can update all packages in one go automatically by doing o update-wrap. This will update the project repository to the latest compatible version of packages, as described in your wrap descriptor file.

Building

One of the things I don’t want to address just yet is building packages. There’s an MSBuild task that makes building the zip files easier, but I don’t want to go down the route Horn got and turn a dependency manager into dependency hell. So for now, people are expected to build their own zip files. It’s only temporary however, as I do expect in the next few versions for OpenWrap to not be built from a build script anymore, which may end up being the default convention. Who knows? For now, OpenWrap integrates itself in your existing projects by replacing the target import by openwrap’s one.

Installing…

I’ve not put up binary releases just yet, but when they are made available (or if you build it yourself), the shell is able to automatically install itself and retrieve the latest version of packages that it needs to run. That means that the installer is 35kb. You shouldn’t have much difficulty downloading o.exe and running with it very quickly.

Going forward

There’s a lot more to OpenWrap so far, from anchoring to the build system, but this is enough of an intro to give you a taste of what I’ve been up to for a few months.

Bar one piece of functionality on the client, which I’m in the middle of changing around (anchoring, a blog post at some point will explain what that is), we’re nearly ready for a first beta release.

Want to contribute? The site is at http://github.com/openrasta/openwrap. While it’s under the OpenRasta brand, the project is stand-alone, and I will be managing OpenWrap as an independent tool. I will however base any further OpenRasta developments on it.

I know there has been recently a lot of noise from many parties in the package management space. The only thing I can tell you is that I’m fully committed to bringing OpenWrap in the hands of developers as quickly as possible. It’s a different approach to a complicated problem, and I think it’s hitting a damn nice spot even if I may say so myself. I’ll post in a few days the first binaries (when the server is up) so you can start playing with it, open accounts and start publishing your own wraps. After that, there’s a lot of work to do to enable testing, documentation and many other scenarios.

One last word

If I was to request one last thing from you, reader, it would be the following. Package management is going to be the next Vietnam war on .net, and many people will come up with different solutions. It’s all for the best, and most projects, as they have done many times before, will die off on their own. This is to be expected. As such, I implore the .net world to let enough time for each of those projects to compete purely on the quality and functionality they provide, and let the best one eliminate naturally the other ones. Don’t succumb to the belief that standardizing early will help, or that popularity would be a distinctive factor. If we fuck-up the package management story on .net, we may never recover, so give it a bit of time and be circumspect in your analysis.

And don’t forget, my project got a cooler name anyway.

:)

12 comments:

John said...

You should aim to have this released in the Web PI. What do u think?

Troy said...

Thanks for the intro Sebastien - I've been waiting for a package mgmt tool in .Net _forever_ and even took a stab at it myself for a while with a (now long gone) project I called ngems. There is certainly a ton of effort that needs to be put forth to achieve success, and it looks like you're already well on your way.

One thing that I didn't see a lot about in this post is non-assembly exports. As an example, it would be great to be able to offer up an entire MVC Area as an export alongside the assemblies that area depends upon.

Also, once I've installed a wrap into my project, does it automatically update my csproj file with the correct reference to the openwrap cache, or do I need to manually add the reference? If it is automatic, how does openwrap know which project in the solution to target?

Krzysztof Koźmic said...

@John

PI sucks to be honest. The only upside of having a release there is that it's Microsoft so will obviously get a lot of eyeballs that may stumble upon your project but other than that I see little value in it.

Mark said...

wraps.openwraps.org can not be resolved. Is this a known problem?

Sebastien Lambla said...

@John - Yes, that's doable, not sure it has much value, but as Krzysztof says, it adds in visibility so why not. Packages themselves can be shown in the PI, so that's something I've considered too.

@Troy - Absolutely. Binaries are just one export type, there's going to be more (documentation, static web content with js / css files come to mind first). It should be absolutely possible to have an aspnet-mvc export, a fubu-mvc one and an openrasta one if needed.

The csproj don't need to be updated, OpenWrap is part of the build in OpenWrap consumers, so any change to your dependency file autoamtically links all the projects that are consuming the wraps. As for which projects, it adds the references to all of them. If they're not used, they won't be loaded and it won't matter, if they're used, then all is good and the assemblies will be resolved in whatever way assemblies are resolved for your destination package. THat's another blog post coming up.

@Mark - Yes, the server wraps.openwrap.org is not up yet to general consumption, I'm continuing my dogfooding on the remote side this week and will unleash the beast this weekend if all goes well.

R said...
This comment has been removed by the author.
R said...

Now the dependency resolving abstraction being lifted away from visual studio projects is absolutely interesting. I make a reference to the wrap, the wrap proxy automatically keeps me up to date with the latest version of the dependency. That's a freakin killer feature that may solve the biggest problem in .NET framework which is managing the "it must be this and only this assembly version" for dependencies.

I'm one of the Nu (#nuproj) guys, probably the loudest one right now (@ferventcoder). But like I said, I'll be watching OpenWrap. To me, outside of gems, it seems on the surface to be the most compelling product and nu's biggest competitor. I'm pretty excited for the future of .NET OSS!

Sebastien Lambla said...

We'll see what the future holds. Healthy competition is good. As I explained to a bunch of people, what's really important is not which format the packages are in, but that the packages exist at all. Moving from one format to the other should be trivial, so should be pushing both formats, once you understand your dependencies well. :)

林聖瑤 said...

快樂,是享受工作過程的結果..................................................

李淑成李淑成 said...

唯有穿鞋的人,才知道鞋的哪一處擠腳............................................................

Troy Goode said...

I see you mentioned "static web content", which is certainly more what I'm getting at, but you also say no modifying of the csproj file will be necessary - I'm a bit confused at what OpenWrap will actually do with the static web content in that case.

I work on an OSS project called the Mvc Membership Starter Kit. It consists of two things:

1) Interfaces & wrapper classes that wrap the built-in MembershipProvider, RoleProvider, and FormsAuthentication classes, allowing you to access these in a mockable way for testing.

2) An MVC area with a controller and several views, that gives you a simple User/Role Administration screen out of the box that works with the providers.

What I'd ideally like is the ability to not only let developers bring down the assemblies, documentation, and miscellaneous files, but to ALSO go ahead and "install" the User Administration area into their MVC project.

Perhaps I would do this by exporting an OpenWrap command that does the "installation"?

Sebastien Lambla said...

@Troy: How you implement that would depend on what you want to achieve.

When I talk about static content, I'm thinking of a package that any web framework can use. How they get the files in is dependent on the project: The package would probably be declared as anchored, which means you'd be able to link to it manually by going to /wraps/anchored/jquery and get whatever files are in there.

Alternatively, someone could build a VirtualPathProvider that would redirect all content in /web-static to a /lib folder, so that any asp.net site could use the content.

Or maybe for fx that understand OpenWrap (like openrasta), that linking will be done out of the box.

Or maybe we could change the project using the same injection process to add the files in a known location at the msbuild level.

Or maybe one could use post-install hooks to go and add stuff to a project file themselves, whenever something is installed.


There's many ways to achieve the same thing, and at this stage I don't have the resources to be prescriptive about it, I hope and expect the community to take ownership.

Post a Comment