I’ve been following Photosynth for a while now (amazingly nearly 2 years!), since seeing some early demos of Piazza San Marco in Venice to the announcement that Microsoft finally unleashed Photosynth upon the web (see article). At the last Silverlight User Group meeting, we had a talk from John Penrose about Graphico‘s efforts of synthing the London Eye; and this, coupled with my passion for photography and a upcoming trip to Hong Kong made me think of exploring the latest version of Photosynth (rather than the tech previews).

Photosynth provides the avid photographer with the opportunity to present photos as a relational composition – the end result is a 3D looking presentation of the images in the Photosynth composition, however since the images are actually the presented in their 2D slices, I like to think that its rather more 2½D!

It is presented as a Silverlight application that leverages Deep Zoom to keep the image quality respectable to the area being actually viewed (allowing you to get 1:1 to large resolution images without the hefty download).

Photosynth looks like this below… (I did NOT prepare this one):

Photosynth 2½ view example

Maybe this is a good time to point out a rather fascinating feature – point clouds. These is the mesh to which Photosynth determines where corresponding points in the composition overlap. You can see the point cloud by pressing "P" and the results can be stunning… the more overlapping points there are in a composition the more spectacular is the resulting 3D model. You can see the corresponding point cloud for the synth above, in the picture below:

 Photosynth point cloud example

That is enough of an intro to Photosynth, what I really wanted to share was the results of a bit of experimenting with Photosynth. I just recently returned from vacation in Hong Kong – its an absolute joy taking photos there especially with the diverse array of subject matter, from culture/traditions, architecture/landscapes, to everyday people and family. I had planned to visit the Sha Tin Racecourse (one of the two horse racing venues in Hong Kong) and thought that this would be an interesting panorama to synth.

Normally, when I’ve taken panoramas I’ve had to resort to taking a series of panning photos and then stitching them together into a larger image. I’m sure that you know the sort of picture that you end up with… it kind of looks like the to images below: 

Sha Tin Panorama #1 @9% Sha Tin Panorama #2 @9%

Urrghh…. Although this is my normal method of stitching panned panoramas, I loathe those horrible black background fills where the lens distorts, and those "seams" where the stitching software tries to overlap the images. There are a number of tolerance settings that can be tweaked, but it takes time to get a decent seamless image that one can be happy with.

Now, I shoot with my DSLR which results in 3888×2592 sized images (~2MB each) – and to be honest is a bit big for posting on your average web site (of course Flickr and suchlike can help you out). Let me bring to your attention, that processing a collection of these images into a panorama is usually results in a grotesquely large file for the web… the two stitches above are reduced to 9% – yes! 9%! – since the full stitches are 11469×2729 (6MB) and 7572×2708 (3.2MB).

Roll in Photosynth

With my Windows Live account in hand, creating a Photosynth space turned out to be really easy. It will download a tool to your computer to facilitate the processing and publishing of your synth. It’s all very straightforward and there is plenty of help available.

I had taken about 200+ images throughout the day at the Sha Tin Racecourse, but for the purpose of these exercises I had selected about 50 images and 4 panoramic stitches. Many of these photos were actually from the same vantage point, just generally panning and scanning the surrounding area as I normally do – I also selected a few images from my wanderings about the racecourse too. Let me state here that, normally I do a fair amount of post processing to my photos and these have not been post processed otherwise I would be posting article weeks later!

Resisting a big bang approach, I selected 24 initial images that looked both interesting and what I "expected" to be reasonably suitable for my first synth. Wow, without even trying I managed to hit a 96% synth rate (which means that the Photosynth tool was able to match the majority of my images to the generated point cloud). The "unsynthed" 4% would site in their own point clouds if they could be grouped further. I don’t need to list the images themselves, the synth itself can show you by pressing the "@" key (UK keyboard, since the US keyboard has "~" instead) and then you can Deep Zoom from a neatly rendered high level view into the detailed image I imported.

After playing about with the synth, exploring the different angles/depths and getting a general feel for the navigation, I thought that maybe there was a better way of controlling the composition of the synth. I picked a even smaller control set of 14 related images and just added them in filename order (just a simple CTRL-A and ‘OK’) and was surprised that the synth rate had dropped to 58%. I imagine that Photosynth works best with either lots of images or at least those with good overlap – when I shoot panoramic compositions I do tend to reduce the overlapping margins so that I don’t end up with many "stitch seams" – so I’ll have to change my behaviour for future synthing. Using the same 14 images as before, but importing them in a random order, resulted in a drop in the synth rate which would suggest that when Photosynth is processing files for the cloud map, it does it in a sequential order expecting there to be a reasonable level of similarity between successive images. The previous synth worked pretty well since there were groups of related images already in sequence that I took the pictures.

Forming a notion of what makes the Photosynth tool construct the point cloud, I used the 14 images again but added them in my own "visual order", the result this time was a more successful 65% synth rate giving some proof to my observations that similar images should be added sequentially. Determined by getting an even higher synth rate out of the 14 images, I turned my attention to overlap. If Photosynth relies on a decent amount of overlap, then maybe, if I add in my whopping 13121×2660, 7.3MB panoramic stitch image which features at least 5 of the individual images, then this might get my synth rate up. This worked a treat and achieved a 100% synth rate, so you are able to navigate to all pictures within the synth. Using the stitched panorama comes at a disadvantage though because overall experience of the synth has been greatly reduced, since Photosynth displays the panoramic stitch in the initial view and makes it difficult/awkward to get round it. You win some and you lose some!

My endeavours have been summarised in the following table… including links to all the synths created.

Photosynth Name

Images

Synth Result

Conditions

Blog Test #1

24 96% Initial test of compiling a synth without any panoramic stitch images.

Blog Test #2

14 58% Reduced set of photos from Blog Test #1, added in filename sequence.

Blog Test #3

14 41% Same set as Blog Test #2 but added in a random order.

Blog Test #4

14 65% Same set as Blog Test #2 but added in a visually optimised order.

Blog Test #5

15 100% Same set as Blog Test #2 but added in filename sequence AND with a panoramic stitch image.

Blog Test #6

46 91% Same set as Blog Test #2 with additional photos – all added in filename order.

Sha Tin Racecourse

46 98% Same as Blog Test #6 but added in a visually optimised order.

The last tests I performed were to add more photos to my previous set of control images, resulting in a total of 46 normal images with no panoramic stitches. The results we pretty good at 91%, but armed with my new found knowledge of how to order the imported images, I created my final synth by adding them in a visually optimised order.

You can see the final fruits of my labour, below (if you look carefully you’ll find the horses galloping past!):

Embedding a synth as I’ve done above is easy, since each synth allows you to export a unique HTML snippet so that you can embed them to your own web site:

<iframe style="border: black 4px solid;"
        src="http://photosynth.net/embed.aspx?cid=64bfdb94-35af-437b-a201-277e2dd8d9be"
        frameborder="0" width="600" height="480"></iframe>

What next….

Well, the Photosynth team have a regularly updated blog which is a veritable mine of information on how to get the most out of your synthing efforts. What I did not know was that they published a tricks and tips guide not more than 2 days ago! I wish that I had been forearmed with this guide since it would have made me slightly modify my approach to composing the pictures to make up the Photosynth. Ensuring that overlapping your images by quite a wide margin was one of their top tips that I found out the hard way, but I’m pretty glad that most of the rules are straightforward and won’t make me readjust my photography technique in a major way – I can continue taking my usual photos and still get some good Photosynth opportunities at the same time.

Now go and take loads of photos and get going with Photosynth!

mark.


 UPDATED! I have added more information to my synths… keywords, Virtual Earth map tags..

I’m sure that the Photosynth team will be expanding more "attributes" that we can hang off our synths too. 🙂

Map tags are particularly cool… look for the map icon icon which will popup a mini Virtual Earth window for you to locate where your synth related to in the world.

To find and view mapped synths, go to Virtual Earth, type in a location eg: "Sha Tin, Hong Kong SAR", and then click "Explore collections" which will change the left panel to display a variety of items such as related photos, 3D models or synths in the map area.

Virtual Earth search results

You can filter down to just the Photosynth items by clicking the right-most filter icon (highlighted below).

Virtual Earth Photosynth results

mark.

 

This is a cross post from my EMC blog, mainly for backup duplicity and to aggregate some of my past postings. My EMC blog used to be under the Conchango brand but was acquired by EMC so I’ve also retrospectively refreshed some of the old links and maybe a tweak a bit of content too.

permalink to the original post here