Calligraphic Packing Jie Xu
Craig S. Kaplan
Computer Graphics Lab David R. Cheriton School of Computer Science University of Waterloo GI'07 May 28, 2007
Outline 1
Background of NPR Packing Artistic Packing Text Packing Challenge of Calligraphic Packing
2
Approach The procedure of our system Container extraction Container subdivision Letter Warping
3
Results and conclusion
Artistic Packing Representing a large image from smaller, recognizable elements. It has been explored by many artists.
Giuseppe Arcimboldo
Sandro Del-Prete
Image Packing
Hausner, Simulating decorative mosaics, SIGGRAPH 2001. Dalal et al., A spectral approach to NPR packing, NPAR 2006. Kaplan and Salesin, Escherization, SIGGRAPH 2000.
NPR packing
Kim and Pellacini, Jigsaw image mosaics, SIGGRAPH 2002. Gal et al., Non-realistic expressive modeling, SIGGRAPH 2006 sketch.
Transfer to Text
Use text to tile a shape. Letters should be legible. Letters can stand a signi cant amount of deformation.
Transfer to Text
Jigsaw image mosaics: only support minor deformation. Decorative mosaics and spectral packing: aim at packing a large number of small elements without deformation. Escherization: tile a plane with a lot of copies of one deformed shape strictly. Expressive modeling: compose 3D shapes with rigid motion.
Islamic Calligraphy
by Hassan Musa
Representational Calligraphy
by AlmapBBDO
Representational Calligraphy
Calligraphic Packing It is a combination of calligraphy and packing. Given a region and a sequence of letters, construct a non-overlapping arrangement of deformed glyphs. The glyphs ll the region as much as possible. Glyphs are recognizable. The arrangement should follow the order of these letters.
Algorithm Convert an image into a container. Subdivide image into regions. Warp the letters into these regions.
elephant
Container Extraction
Use graph-cut algorithm to remove background. Apply Gaussian blur to smooth the image. Threshold the image to produce a bi-level result.
(a)
(b)
(c)
(d)
Subdivision
Set the starting arrangement of letters. Run a level-set algorithm to grow letters and cluster pixels. Use Lloyd's method to create an even arrangement.
Convert Regions into Paths
Smooth the boundaries of regions by morphological operations. Trace the boundary to extract paths.
Warping Create a mapping from the convex hull of glyph to the subregion. Given the convex hull Ci of glyph and the subregion Ri . Do convex partition for the subregion. Place the same number of sample points around Ci and Ri . Select a correspondence. Create subdivision for the convex hull. Do warping in each convex piece.
Warping Create a mapping from the convex hull of glyph to the subregion. Given the convex hull Ci of glyph and the subregion Ri . Do convex partition for the subregion. Place the same number of sample points around Ci and Ri . Select a correspondence. Create subdivision for the convex hull. Do warping in each convex piece.
Warping Create a mapping from the convex hull of glyph to the subregion. Given the convex hull Ci of glyph and the subregion Ri . Do convex partition for the subregion. Place the same number of sample points around Ci and Ri . Select a correspondence. Create subdivision for the convex hull. Do warping in each convex piece. X Y D
D E
@ A
X Y
[
[
Z
Z
B
C
. /
. /
* +
0 1
V W
V W
!
, -
\ ]
"
" #
( )
2 3 R S
2 3
R S
>
> ?
4 5
4 5
T U
< =
F G
$ %
^ _
^ _
H I
c : ;
J
J K
8 9 L
L
M
M
` a
P Q
N
N
O
O
6 7
6 7
& '
b
b
Warping Create a mapping from the convex hull of glyph to the subregion. Given the convex hull Ci of glyph and the subregion Ri . Do convex partition for the subregion. Place the same number of sample points around Ci and Ri . Select a correspondence. Create subdivision for the convex hull. Do warping in each convex piece.
[
X Y D E
D E
@ A
Z
Z
B
C
. /
* +
0 1
V W
!
, -
\ ]
f g
"
" #
( )
f g
2 3
2 3
2 3 R
2 3
R S
>
d e
d e
d e
d e
> ?
4 5
4 5
T U
< =
F G
$ %
^ _
^ _
H I
c
c
b
b
: ;
J K
J K
8 9 L
L
M
M
` a
P Q
N
N
O
O
6 7
6 7
& '
Warping Create a mapping from the convex hull of glyph to the subregion. Given the convex hull Ci of glyph and the subregion Ri . Do convex partition for the subregion. Place the same number of sample points around Ci and Ri . Select a correspondence. Create subdivision for the convex hull. Do warping in each convex piece.
Warping Create a mapping from the convex hull of glyph to the subregion. Given the convex hull Ci of glyph and the subregion Ri . Do convex partition for the subregion. Place the same number of sample points around Ci and Ri . Select a correspondence. Create subdivision for the convex hull. Do warping in each convex piece.
Warping Create a mapping from the convex hull of glyph to the subregion. Given the convex hull Ci of glyph and the subregion Ri . Do convex partition for the subregion. Place the same number of sample points around Ci and Ri . Select a correspondence. Create subdivision for the convex hull. Do warping in each convex piece.
Geometric Shape Cost Use \shape context" to measure the similarity of two shapes. Compute a log-polar histogram for every reference point. Compute the geometric similarity 4g as the sum of histogram distance between all pairs of points. < =
?
?
>
>
@ A
@ A
H I
B
B C
¡
F G
:
F G
J K
J K
: ;
D
D E
L M
L M
N O
`
` a
b
N O
b
c
P
P Q
d e
d e
R S
R S
À Á
À Á
À Á
4 5
À Á : ;
4 5
À Á : ;
À Á : ;
< =
@ A
< =
@ A
> ?
@ A
> ?
> ?
B C D E
6 7
B C D E
6 7
6 7
F G
8 9
F G
8 9
8 9
ü Hý I
J K
ü Hý I
J K
J K
L M
L M
4 5
À Á : ;
4 5
À Á : ;
À Á : ;
< =
@ A
< =
@ A
> ?
@ A
> ?
> ?
B C D E
6 7
B C D E
6 7
6 7
F G
8 9
F G
8 9
8 9
ü Hý I
J K
ü Hý I
J K
J K
L M
L M
4 5
À Á : ;
4 5
À Á : ;
À Á : ;
< =
@ A
< =
@ A
> ?
@ A
> ?
> ?
B C D E
6 7
B C D E
6 7
6 7
F G
8 9
F G
8 9
8 9
ü ý H I
J K
ü ý H I
J K
J K
L M
L M
4 5
À Á : ;
4 5
À Á : ;
À Á : ;
< =
@ A
< =
@ A
> ?
@ A
> ?
> ?
B C D E
6 7
B C D E
6 7
6 7
F G
8 9
F G
8 9
8 9
ü Hý I
J K
ü Hý I
J K
J K
L M
L M
4 5
À Á : ;
4 5
À Á : ;
À Á : ;
< =
@ A
< =
@ A
> ?
> ?
B C D E
6 7
B C D E
6 7
6 7
F G
8 9
F G
8 9
8 9
ü Hý I
J K
ü Hý I
J K
J K
L M
L M
4 5
À Á : ;
4 5
À Á : ;
À Á : ;
< =
@ A
< =
@ A
> ?
> ?
B C D E
6 7
B C D E
6 7
6 7
F G
8 9
F G
8 9
8 9
ü ý H I
J K
ü ý H I
J K
J K
L M
L M
 à !
 à !
 à !
" #
$ %
" #
$ %
& '
& '
Þ ß ( )
* +
Þ ß ( )
* +
* +
, -
ò ó
, -
ò ó
ò ó
þ .ÿ /
0 1
þ .ÿ /
0 1
0 1
2 3
2 3
 à !
 à !
 à !
" #
$ %
" #
$ %
& '
& '
Þ ß ( )
* +
Þ ß ( )
* +
* +
, -
ò ó
, -
ò ó
ò ó
þ .ÿ /
0 1
þ .ÿ /
0 1
0 1
2 3
2 3
 à !
 à !
 à !
" #
$ %
" #
$ %
& '
& '
Þ ß ( )
* +
Þ ß ( )
* +
* +
, -
ò ó
, -
ò ó
ò ó
þ .ÿ /
0 1
þ .ÿ /
0 1
0 1
2 3
2 3
 à !
 à !
 à !
" #
$ %
" #
$ %
& '
& '
Þ ß ( )
* +
Þ ß ( )
* +
* +
, -
ò ó
, -
ò ó
ò ó
þ .ÿ /
0 1
þ .ÿ /
0 1
0 1
2 3
2 3
 à !
 à !
 à !
" #
$ %
" #
$ %
& '
& '
Þ ß ( )
* +
Þ ß ( )
* +
* +
, -
ò ó
, -
ò ó
ò ó
þ .ÿ /
0 1
þ .ÿ /
0 1
0 1
2 3
2 3
Ä Å º »
& '
& '
Þ ß ( )
* +
Þ ß ( )
* +
* +
, -
ò ó
, -
ò ó
ò ó
þ .ÿ /
0 1
þ .ÿ /
0 1
0 1
2 3
Ò Ó
Ò Ó
à á Ø Ù
æ ç
à á Ø Ù
æ ç
æ ç
ì í
ô õ
ì í
ô õ
ô õ
Ò Ó
Ò Ó
à á Ø Ù
æ ç
à á Ø Ù
æ ç
æ ç
ì í
ô õ
ì í
ô õ
ô õ
Ò Ó
Ò Ó
à á Ø Ù
æ ç
à á Ø Ù
æ ç
æ ç
ì í
ô õ
ì í
ô õ
ô õ
Ò Ó
Ò Ó
à á Ø Ù
æ ç
à á Ø Ù
æ ç
æ ç
ì í
ô õ
ì í
ô õ
ô õ
Ò Ó
Ò Ó
à á Ø Ù
æ ç
à á Ø Ù
æ ç
æ ç
ì í
ô õ
ì í
ô õ
ô õ
Ò Ó
à á Ø Ù
æ ç
à á Ø Ù
æ ç
ô õ
ì í
ô õ
f g
f g
h
° ±
h i
T U
j k
j k
V W
V W
l m
l m
X Y
> ?
> ?
& '
@ A
@ A
$ %
X Y
& '
$ %
® ¯
n
Z [
Ä Å º »
Ä Å º »
Ì Í
Ì Í
 Ã
 Ã
!
!
" #
$ %
" #
$ %
Ä Å º »
Ä Å º »
Ä Å º »
Ì Í
Ì Í
Ä Å º »
Ä Å º »
Ä Å º »
Ì Í
Ì Í
Ä Å º »
Ä Å º »
Ä Å º »
Ì Í
Ì Í
Ä Å º »
Ä Å º »
Ä Å º »
Ì Í
Ì Í
Ä Å º »
Ä Å º »
Ä Å º »
Ì Í
Ì Í
Ä Å º »
Ä Å º »
Ä Å º »
Ì Í
Ì Í
 Ã
& '
& '
$ %
$ %
n o
!
& '
$ %
Ì Í
& '
$ %
à á
à á
2 3
Ò Ó
Ì Í
p q
\
log r
\ ]
² ³
r s
¨ ©
^ _
^ _
t u
Ò Ó
Ò Ó
Ò Ó
Ò Ó
Ì Í
Ì Í
Ì Í
Ì Í
Ì Í
Ò Ó
æ ç
ì í
ô õ
Æ Ç ¼ ½
Æ Ç ¼ ½
Î Ï
Î Ï
Ô Õ
Ú
Ô Õ
â ã Ú Û
è é
â ã Ú Û
è é
î
è é
î ï
ö ÷
î ï
ö ÷
ö ÷
Æ Ç ¼ ½
Æ Ç ¼ ½
Î Ï
Î Ï
Ô Õ
â Ú
Ô Õ
â ã Ú Û
è é
â ã Ú Û
è é
î
è é
î ï
ö ÷
î ï
ö ÷
ö ÷
Æ Ç ¼ ½
Æ Ç ¼ ½
Î Ï
Î Ï
Ô Õ
â Ú
Ô Õ
â ã Ú Û
è é
â ã Ú Û
è é
î
è é
î ï
ö ÷
î ï
ö ÷
ö ÷
Æ Ç ¼ ½
Æ Ç ¼ ½
Î Ï
Î Ï
Ô Õ
â ã Ú Û
è é
â ã Ú Û
è é
î
è é
î ï
ö ÷
î ï
ö ÷
ö ÷
Æ Ç ¼ ½
Æ Ç ¼ ½
Î Ï
Î Ï
â ã Ú Û
è é
â ã Ú Û
Æ Ç ¼ ½
Æ Ç ¼ ½
Î Ï
Î Ï
¸ ¹
¸ ¹
¸ ¹
È É ¾ ¿
Ê Ë
È É ¾ ¿
Ê Ë
Ê Ë
Ð Ñ
Ð Ñ
¸ ¹
¸ ¹
¸ ¹
È É ¾ ¿
Ê Ë
È É ¾ ¿
Ê Ë
Ê Ë
Ð Ñ
Ð Ñ
¸ ¹
¸ ¹
¸ ¹
È É ¾ ¿
Ê Ë
È É ¾ ¿
Ê Ë
Ê Ë
Ð Ñ
Ð Ñ
¸ ¹
¸ ¹
¸ ¹
È É ¾ ¿
Ê Ë
È É ¾ ¿
Ê Ë
Ê Ë
Ð Ñ
Ð Ñ
¸ ¹
¸ ¹
¸ ¹
È É ¾ ¿
Ê Ë
È É ¾ ¿
Ê Ë
Ê Ë
Ð Ñ
Ð Ñ
¸ ¹
¸ ¹
¸ ¹
È É ¾ ¿
Ê Ë
È É ¾ ¿
Ê Ë
Ê Ë
Ð Ñ
Ð Ñ
â
t u
Ò Ó
8 9
Ô Õ
â Ú
v w
0
6 7
x y
4 5
Ú
Ô Õ
è é
î
è é
î ï
ö ÷
î ï
ö ÷
ö ÷
Ü
Ö ×
ê ë
ð
ê ë
ð ñ
ø ù
ð ñ
ø ù
ú
ø ù
Ô Õ
â Ú
Ô Õ
â ã Ú Û
è é
â ã Ú Û
è é
î
è é
î ï
ö ÷
î ï
ö ÷
ö ÷
Ö ×
ä å Ü Ý
ê ë
ä å Ü Ý
ê ë
ð
ê ë
ð ñ
ø ù
ð ñ
ø ù
ø ù
ú û
ú û
Ö ×
ä å Ü Ý
ê ë
ä å Ü Ý
ê ë
ð
ê ë
ð ñ
ø ù
ð ñ
ø ù
ø ù
ú û
ú û
Ö ×
ä å Ü Ý
ê ë
ä å Ü Ý
ê ë
ð
ê ë
ð ñ
ø ù
ð ñ
ø ù
ø ù
ú û
ú û
Ö ×
ä å Ü Ý
ê ë
ä å Ü Ý
ê ë
ð
ê ë
ð ñ
ø ù
ð ñ
ø ù
ø ù
ú û
ú û
Ö ×
ä å Ü Ý
ê ë
ä å Ü Ý
ê ë
ð
ê ë
ð ñ
ø ù
ð ñ
ø ù
ø ù
ú û
ú û
Ö ×
ä å Ü Ý
ê ë
ä å Ü Ý
ê ë
ð
ê ë
ð ñ
ø ù
ð ñ
ø ù
ø ù
ú û
ú û
Ü Ý
ê ë
Ü Ý
ú û
ú û
2 3
¬
Ô Õ
â
Ö ×
0 1
6 7
x y
Ö ×
ä Ü
ú
¬
Ö ×
ä Ü
ú
z {
~
~
Ö ×
ä Ü
ª «
|
. /
Ö ×
Ö ×
ä Ü
Ö ×
ä Ü
ä Ü
, -
+
!
)
´ µ
¦ §
¦ §
"
" #
$
¤ ¥
¶ ·
¢
¢ £
'
'
&
&
$
%
¤
ú
(
*
ú
ú
. /
| }
θ
ú
Orientation Cost
Orientation is important to preserve legibility.
Use a least-square method to compute the rigid motion from original glyph to warped one. From the rotation angle , we de ne the orientation cost: 4o = =.
Area Cost
Warped glyphs should ll the subregions as much as possible. Area cost is de ned as: 4a = 1 Aw =Ar .
Total Cost
Shape matching cost is
cost=0.328
4 = 4g + 4o + 4a .
cost=0.369
cost=0.371
cost=0.429
cost=0.512
Warping Multiple Typefaces For each letter, we warp lowercase and uppercase glyphs from multiple typefaces.
cost=0.324 cost=0.396 cost=0.333 cost=0.376
cost=0.469 cost=0.456
cost=0.36
cost=0.463
Rendering Styles Perturb boundaries of letters with random osets. Fill letters with streamlines.
Rendering Styles Perturb boundaries of letters with random osets. Fill letters with streamlines.
freedom & slavery
successful
Results
Results
monalisa
Results
monalisa
lose & win
Results
Results
niao (bird)
Results
niao (bird)
muse
Results
Results
laugh & cry
Results
laugh & cry
graceful
Results
laugh & cry
graceful
Da Vinci code
Conclusion and Future Work
Distribute letters automatically. Improve the letter deformation model.
Questions?
User-Speci ed Subdivision User-speci ed clustering.
User-speci ed exclusion.