SfS++

Structure-from-Sherds++: Robust Incremental 3D Reassembly of Axially Symmetric Pots from Unordered and Mixed Fragment Collections

Seong Jong Yoo^{*, 1}, Sisung Liu^{*, 2}, Muhammad Zeeshan Arshad³,
Jinhyeok Kim⁴, Young Min Kim⁵, Yiannis Aloimonos¹, Cornelia Fermüller¹,
Kyungdon Joo⁴, Jinwook Kim^{†, 3}, Je Hyeong Hong^{†, 2}
¹University of Maryland, ² Hanyang University, ³ KIST, ⁴ UNIST, ⁵ Seoul National University
* Equal contribution
† Co-Corresponding author

Abstract

Reassembling multiple axially symmetric pots from their constituent fragments is a crucial task for preserving cultural heritage, often requiring extensive manual effort from restoration experts. Previous approaches framed this as a global puzzle-solving problem, either optimizing all potential fragment pairs simultaneously or training models on large datasets to find the best configuration. However, these methods suffer from false positive matches between indistinctive sharp fracture surfaces in pot fragments, leading to local minima solutions or scalability issues in multiple pots reassembly scenarios. To tackle this problem, we propose an efficient incremental method for multiple axially symmetric pots, inspired by Structure-from-Motion approaches for 3D reconstruction. Our method, Structure-from-Sherds++ (SfS++) resolves indistinguishable false matches with beam search, exploring various registration options. Multiple reconstruction graphs in a state enable simultaneous reassembly, merging between reconstruction graphs for a base-agnostic and efficient process. Our method achieves 87 % reassembly accuracy on a dataset of 142 real fragments from 10 pots. It outperforms other methods in handling complex fracture patterns with mixed datasets and achieves sate-of-the-art results.

Fig. 4 (a) Overall pipeline: The SfS++ pipeline consists of three modules: Feature Extraction (FE module), Pairwise Matching (PM module), and iterative Base-Agnostic Incremental ShErd Registration (BAISER module). (b) BAISER submodules: The BAISER module utilizes a *multi-graph* beam search algorithm, which iteratively repeats two components: Exploring candidates for expansion and State expansion. In the State Expansion components, the red dotted line represents graph merging.

Dataset

Total 10 different potteries with 142 fragments. Scanned from real-broken sherds (not in the real-scale)

Results

* Visualization with downsampled mesh. For the original quality results, please visit here

Single Object

SfS++ Dataset

Pot A

Type: Dish
# Sherds: 8
# Edges: 15
Acc. : 100 %

Pot B

Type: Dish
# Sherds: 9
# Edges: 15
Acc. : 100 %

Pot C

Type: Tea pot
# Sherds: 4
# Edges: 5
Acc. : 100 %

Pot D

Type: Vase
# Sherds: 29
# Edges: 69
Acc. : 82.1 %

Pot E

Type: Vase
# Sherds: 31
# Edges: 62
Acc. : 100 %

Pot F

Type: Dish
# Sherds: 6
# Edges: 7
Acc. : 100 %

Pot G

Type: Dish
# Sherds: 9
# Edges: 15
Acc. : 100 %

Pot H

Type: Dish
# Sherds: 11
# Edges: 17
Acc. : 90.9 %

Pot I

Type: Vase
# Sherds: 27
# Edges: 65
Acc. : 92.6 %

Pot J

Type: Tea pot
# Sherds: 11
# Edges: 21
Acc. : 72.7 %

Breaking Bad

Plate 1

# Sherds: 12
# Edges: 22
Acc. : 91.7 %

Plate 2

# Sherds: 8
# Edges: 12
Acc. : 100 %

Plate 3

# Sherds: 7
# Edges: 12
Acc. : 100 %

Bowl 1

# Sherds: 10
# Edges: 11
Acc. : 80.0 %

Bowl 2

# Sherds: 11
# Edges: 12
Acc .: 90.0 %

Bowl 3

# Sherds: 6
# Edges: 11
Acc. : 100 %

Vase 1

# Sherds: 8
# Edges: 12
Acc. : 75.0 %

Vase 2

# Sherds: 9
# Edges: 14
Acc. : 88.9 %

Vase 3

# Sherds: 9
# Edges: 19
Acc. : 100 %

Multiple Objects: Unordered and Mixed

SfS++ Dataset

Experiment 1: All dishes (A + B + F + G + H)

# Objects: 5
# Sherds: 41

# Edges: 64
Accuracy: 97.6 %

Experiment 2: Identical Objects (C + J)

# Objects: 2
# Sherds: 15

# Edges: 26
Accuracy: 80.0 %

Experiment 3: Identical Objects (E + I)

# Objects: 2
# Sherds: 58

# Edges: 127
Accuracy: 96.6 %

Experiment 4: SfS Dataset (A + B + C + D + E)

# Objects: 5
# Sherds: 81

# Edges: 166
Accuracy: 92.5 %

Experiment 5: SfS++ Dataset (A + B + C + D + E + F + G + H + I + J)

# Objects: 10
# Sherds: 142

# Edges: 286
Accuracy: 87.3 %

BibTex

@inproceedings{YooandLiu_2025_SfS,
  title     = {{Structure-From-Sherds++}: Robust Incremental 3D Reassembly of Axially Symmetric Pots from Unordered and Mixed Fragment Collections},
  author    = {Yoo, Seong Jong and Liu, Sisung and Arshad, Muhammad Zeeshan and Kim, Jinhyeok and Kim, Young Min and Aloimonos, Yiannis and Fermüller, Cornelia and Joo, Kyungdon and Kim, Jinwook and Hong, Je Hyeong},
  journal   = {arXiv},
  year      = {2025},
}

@inproceedings{HongandYoo_2021_ICCV,
	author    = {Hong, Je Hyeong and Yoo, Seong Jong and Zeeshan, Muhammad Arshad and Kim, Young Min and Kim, Jinwook},
	title     = {Structure-From-Sherds: Incremental 3D Reassembly of Axially Symmetric Pots From Unordered and Mixed Fragment Collections},
	journal   = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
	month     = {October},
	year      = {2021},
	pages     = {5443-5451}
}

License

SfS++ is available for non-commercial and research use only and may not be redistributed and should follow requirements under the conditions detailed on the license page. If you have any questions, please get in touch with us at yoosj@umd.edu.