I'm sorry I haven't developed Sketch-To-Collage in 12 years... 😅
I'm posting today because Nvidia just announced a tool that it's similar to this concept of sketch-to-collage,
https://techcrunch.com/2019/03/18/nvidia-ai-turns-sketches-into-photorealistic-landscapes-in-seconds/
Although theirs is more "sketch to picture", in the sense that the neural network will always create something that looks plausibly like a photograph. With Sketch-2-Collage you can create any crazy stuff you want.
A friend of mine asked me some years ago to create an iPad app, but doing the same stuff again, in another platform, didn't really motivate me. So I've been programming games and other apps during my free time. Here's a list of all my iOS apps: App Store (David Gavilan).
Sunday, March 24, 2019
Wednesday, October 7, 2009
PhotoSketch
This year two tools similar to Sketch-2-Collage have appeared published in SIGGRAPH. Both are called PhotoSketch, which is quite confusing... Let me briefly introduce them.
PhotoSketch (French one)
This work appeared in a talk in SIGGRAPH 2009 this year, in New Orleans (I was there!). With this PhotoSketch application you can also create image collages searching image databases. The image descriptors are mostly based on shapes, so you trace lines, while in Sketch-2-Collage I decided to give more importance to color. This is because I wanted to replicate the color arrangement of the sketch drawn by the user to that of the final composition. That is the reason why if you feedback the collage to the image search engine you can improve retrieval accuracy, as I show in this paper. The cool stuff of this tool is the user interface. I like that you pain directly over the resulting collage. Perhaps we could do that in Sketch-To-Collage and avoid opening so many windows!
The authors refer to my SIGGRAPH 2007 poster, and claim that their system works with large databases (whatever large means...). I guess they didn't try Sketch-To-Collage, because the image signature is so small that you could fit the signature of millions of images in RAM, and just accessing to it linearly, with any kind of optimization, I get real-time response. Check the response of their system in their video and compare to mine, to see what I mean.
PhotoSketch (Chinese one)
This work is gonna appear in SIGGRAPH Asia 2009 (December, Yokohama, Japan). This system implements almost the same stuff I do for compositing, although the searching part has the same difference I explained in the previous work. As in the French PhotoSketch, they basically use lines to find shapes and you need to add keywords for the searches (in Sketch-To-Collage you can also add keywords). I like how they improved the cutting operation (I use Drag&Drop pasting), but I'm not sure about the improvement of the Poisson image blending operator (I also use a modified version of Poisson blending). The cool thing of this paper is that it automatically creates several plausible collages, so everything is pretty automatic. We could try something similar in Sketch-To-Collage, instead of leaving all the process as interactive. At least have it as an option, and if you don't like it you can always interact with the system. Anyway, this PhotoSketch looks cool but in the video you can't see the system working, just a small part of the painting interface. Their "online demo" is offline, so I can't judge properly at the moment.
Similar as it is to Sketch-To-Collage they could at least put a reference to my work. Well, at least they decided to "steal" the French name, PhotoSketch, and not Sketch-To-Collage ;)
PhotoSketch (French one)
This work appeared in a talk in SIGGRAPH 2009 this year, in New Orleans (I was there!). With this PhotoSketch application you can also create image collages searching image databases. The image descriptors are mostly based on shapes, so you trace lines, while in Sketch-2-Collage I decided to give more importance to color. This is because I wanted to replicate the color arrangement of the sketch drawn by the user to that of the final composition. That is the reason why if you feedback the collage to the image search engine you can improve retrieval accuracy, as I show in this paper. The cool stuff of this tool is the user interface. I like that you pain directly over the resulting collage. Perhaps we could do that in Sketch-To-Collage and avoid opening so many windows!
The authors refer to my SIGGRAPH 2007 poster, and claim that their system works with large databases (whatever large means...). I guess they didn't try Sketch-To-Collage, because the image signature is so small that you could fit the signature of millions of images in RAM, and just accessing to it linearly, with any kind of optimization, I get real-time response. Check the response of their system in their video and compare to mine, to see what I mean.
PhotoSketch (Chinese one)
This work is gonna appear in SIGGRAPH Asia 2009 (December, Yokohama, Japan). This system implements almost the same stuff I do for compositing, although the searching part has the same difference I explained in the previous work. As in the French PhotoSketch, they basically use lines to find shapes and you need to add keywords for the searches (in Sketch-To-Collage you can also add keywords). I like how they improved the cutting operation (I use Drag&Drop pasting), but I'm not sure about the improvement of the Poisson image blending operator (I also use a modified version of Poisson blending). The cool thing of this paper is that it automatically creates several plausible collages, so everything is pretty automatic. We could try something similar in Sketch-To-Collage, instead of leaving all the process as interactive. At least have it as an option, and if you don't like it you can always interact with the system. Anyway, this PhotoSketch looks cool but in the video you can't see the system working, just a small part of the painting interface. Their "online demo" is offline, so I can't judge properly at the moment.
Similar as it is to Sketch-To-Collage they could at least put a reference to my work. Well, at least they decided to "steal" the French name, PhotoSketch, and not Sketch-To-Collage ;)
Introduction
Sketch-to-collage is a tool to aid you in the creation of image collages. You paint simple colored shaped on a canvas, and the system searches for similar images from your digital library (or database) that you can interactively select to create an image collage.
Find the binary file (Java binary) and its source code in the following address:
http://code.google.com/p/sketch2collage/
The purpose of this blog is to keep track of the updates of the program. I chose blogger because it's easier to maintain than other things I've tried. The original blog for this project was hosted in the homepage of my old laboratory: http://www.img.cs.titech.ac.jp/~david/sketch2collage (it seems it's no longer there though...). You can find there some tutorials on using this application. You can also check some collages from the Gallery.
For a brief introduction on how Sketch-To-Collage works, check this video:
The video is from an older version I presented at SIGGRAPH 2007, so I don't show how to input search keywords and other stuff, but it should be self-explanatory. Additional information can also be found in other related publications, or my PhD thesis. Here are some links:
Some history
This project started around 4 years ago, as part of my PhD thesis. The development stopped almost 2 years ago, when I graduated and started working. I was planning to add some features, such as accessing Flickr pictures and such, but since I'm quite busy at work I decided to release the code as Open Source, so others can check it out. It's also to check if there's any interest in the project, so I may continue developing for it.
For other projects and personal information, please visit my web site: http://endavid.com
Find the binary file (Java binary) and its source code in the following address:
http://code.google.com/p/sketch2collage/
The purpose of this blog is to keep track of the updates of the program. I chose blogger because it's easier to maintain than other things I've tried. The original blog for this project was hosted in the homepage of my old laboratory: http://www.img.cs.titech.ac.jp/~david/sketch2collage (it seems it's no longer there though...). You can find there some tutorials on using this application. You can also check some collages from the Gallery.
For a brief introduction on how Sketch-To-Collage works, check this video:
The video is from an older version I presented at SIGGRAPH 2007, so I don't show how to input search keywords and other stuff, but it should be self-explanatory. Additional information can also be found in other related publications, or my PhD thesis. Here are some links:
- David Gavilan, Suguru Saito and Masayuki Nakajima. Sketch-to-collage. In SIGGRAPH '07: ACM SIGGRAPH 2007 posters, Pages 35, ACM Press, New York, NY, USA, 2007. PDF file (1 page), Poster in PDF
- David Gavilan, Suguru Saito and Masayuki Nakajima. Sketch-to-collage: Query-by-sketch based image synthesis. In Proc. IEVC '07, Cairns, Australia, November 2007. PDF file (8 pages)
- David Gavilan, Suguru Saito and Masayuki Nakajima. Query-by-Sketch Based Image Synthesis. IEICE Transactions on Informations and Systems, 2008 E91-D(9). Oxford Journals link
- David Gavilan. Region-Based Image Abstraction for Retrieval and Synthesis. Phd Dissertation, Tokyo, Japan, January 2008. PDF file (214 pages)
Some history
This project started around 4 years ago, as part of my PhD thesis. The development stopped almost 2 years ago, when I graduated and started working. I was planning to add some features, such as accessing Flickr pictures and such, but since I'm quite busy at work I decided to release the code as Open Source, so others can check it out. It's also to check if there's any interest in the project, so I may continue developing for it.
For other projects and personal information, please visit my web site: http://endavid.com
Subscribe to:
Posts (Atom)
