Skip to content

Commit f600783

Browse files
committed
Update Project Page
1 parent 514fbad commit f600783

2 files changed

Lines changed: 66 additions & 24 deletions

File tree

project_page/app/page.tsx

Lines changed: 65 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -375,15 +375,27 @@ export default function VisualSplitPage() {
375375
</div>
376376
</section>
377377

378-
{/* Experiments: Descriptor-to-Image */}
379378
<section id="experiments" className="mb-12">
380379
<h3 className="text-2xl font-semibold mb-4">
381-
Descriptor-to-Image: Visual Examples
380+
Descriptor-to-Image
382381
</h3>
383382
<p className="mb-4 text-sm">
384-
The examples below illustrate how each descriptor contributes to the
385-
reconstruction. Replace the images with your own results placed in{" "}
386-
<code>public/</code>.
383+
<b>VisualSplit</b> treats an image as a composition of three classical, human-interpretable
384+
cues—an edge map for geometry, a segmented colour map for region-wise chroma, and a grey-level
385+
histogram for global illumination—and learns to reconstruct the scene using only these decoupled
386+
descriptors rather than raw RGB. This mask-free pre-training encourages the encoder to capture
387+
globally and locally meaningful structure from “informative absences,” yielding explicit,
388+
disentangled representations.
389+
</p>
390+
<p className="mb-4 text-sm">
391+
The example here illustrates the idea: starting from the original photo (left), we extract the
392+
three descriptors (second column); a human artist, seeing only these cues, can already infer and
393+
sketch the scene (third); our model then recovers a faithful image from exactly the same inputs
394+
(right), demonstrating that the combined descriptors are sufficient for robust reconstruction.
395+
Because each descriptor governs a different attribute—edges ≈ shape, colour segments ≈ region
396+
colours, histogram ≈ brightness—the representation is naturally interpretable and controllable,
397+
enabling independent edits of geometry, colour, or illumination in downstream generation and
398+
editing tasks.
387399
</p>
388400

389401
<div className="flex flex-col md:flex-row items-center md:items-start gap-4">
@@ -437,17 +449,27 @@ export default function VisualSplitPage() {
437449
</figure>
438450
</div>
439451
<p className="mt-2 text-xs text-center">
440-
We thank Jing Dong for the illustration; used with permission.
452+
We thank Jie Dong for the illustration; used with permission.
441453
</p>
442454
</section>
443455

444456
{/* Independent Controls */}
445457
<section id="controls" className="mb-12">
446458
<h3 className="text-2xl font-semibold mb-4">Independent Controls</h3>
459+
<p className="mb-4 text-sm">
460+
<b>VisualSplit</b> separates geometry, region colour, and global illumination into distinct,
461+
human-interpretable inputs—edges, segmented colours, and a grey-level histogram—and trains the
462+
encoder to reconstruct images by integrating these cues. Because edges/colours are treated as
463+
local patch inputs while the histogram conditions the model globally (via cross-attention and
464+
AdaLN-Zero), changing one descriptor steers only its corresponding attribute of the output.
465+
Empirically, varying the histogram alters brightness without affecting shape or region colours,
466+
and swapping the colour map changes chroma while preserving layout and lighting—demonstrating
467+
descriptor-level independence for controllable generation and editing.
468+
</p>
447469
<div className="grid grid-cols-1 md:grid-cols-2 gap-6">
448470
<ImageSlider
449471
title="Illumination (Histogram)"
450-
description={`Modify only the grayscale histogram while keeping edges and colour segmentation fixed. The reconstruction changes global brightness/contrast without altering geometry or region colours.`}
472+
description={`The grey-level histogram summarizes scene brightness and contrast at a global level. Feeding a different histogram (e.g., from low/normal/high-exposure captures) shifts the reconstructed image’s illumination accordingly, while edges and colour segmentation are held fixed—so geometry and regional colour remain stable. This provides a clean “exposure dial” for editing and restoration without touching other attributes.`}
451473
beforeSrc={editImages.illuminationBefore}
452474
options={[
453475
{label: 'Low-light', src: editImages.illuminationLow},
@@ -457,7 +479,7 @@ export default function VisualSplitPage() {
457479
/>
458480
<ImageSlider
459481
title="Colour (Segmentation)"
460-
description={`Re-assign colours to selected segments while keeping edges and histogram fixed. The reconstruction updates region appearance consistently, leaving structure and luminance intact.`}
482+
description={`The segmented colour map assigns coherent regions to colour clusters, letting you recolour specific parts of an image independently of shape or lighting. Replacing only the colour map (e.g., from a re-balanced version of the same photo) changes regional chroma while edges and the global histogram stay intact; editing the map directly enables prompt-free, region-precise recolouring that respects boundaries. `}
461483
beforeSrc={editImages.colourBefore}
462484
options={[
463485
{label: 'f=0.25', src: editImages.colour025},
@@ -472,18 +494,25 @@ export default function VisualSplitPage() {
472494

473495
<section id="applications" className="mb-12">
474496
<h3 className="text-2xl font-semibold mb-4">Applications</h3>
475-
497+
<p className="mb-4 text-sm">
498+
<b>VisualSplit</b> learns decoupled, human-readable controls—edges (geometry), segmented colours
499+
(regional
500+
chroma), and a grey-level histogram (global illumination)—and plugs them into modern image
501+
generators to steer both global layout and local details. In practice, we pair our global
502+
representation with the text embedding (IP-Adapter–style) and inject local representations via
503+
ControlNet into Stable Diffusion 1.5, enabling faithful reconstruction and precise attribute
504+
control
505+
from descriptors alone.
506+
</p>
476507
{/* A. Visual Restoration with Diffusion Models */}
477508
<div className="rounded-2xl border p-4 mb-8">
478509
<h4 className="font-semibold mb-2">A. Visual Restoration with Diffusion Models</h4>
479510
<p className="text-sm mb-3">
480-
VisualSplit provides structured global (<em>histogram</em>) and local (<em>edges</em>, <em>colour
481-
segments</em>)
482-
conditions that plug into Stable Diffusion 1.5. Following the paper, the global
483-
representation can be fed
484-
as an IP-Adapter–style signal while local tokens drive a ControlNet branch, yielding
485-
faithful reconstructions
486-
without relying on prompts.
511+
Using only edge, colour-map, and histogram inputs, our guidance restores images with
512+
substantially higher fidelity than ControlNet, T2I-Adapter, and ControlNet++. This shows
513+
that
514+
descriptor-structured conditions recover both global appearance and fine structure more
515+
reliably than raw controls.
487516
</p>
488517
{/* Descriptor inputs */}
489518
<figure className="mx-auto rounded-xl border p-2 md:w-1/3 flex flex-col items-center">
@@ -500,21 +529,28 @@ export default function VisualSplitPage() {
500529
gt={appAssets.restorationGT}/>
501530

502531
{/* Metrics table (from Table 3) */}
503-
<RestorationTable />
532+
<RestorationTable/>
504533
</div>
505534

506535
{/* B. Descriptor-Guided Editing (No retraining) */}
507536
<div className="rounded-2xl border p-4 mb-8">
508537
<h4 className="font-semibold mb-2">B. Descriptor-Guided Editing (No Retraining)</h4>
509538
<p className="text-sm mb-3">
510-
Editing is performed by modifying individual descriptors while keeping others fixed. Below
511-
we show two independent edits: altering the <em>colour segmentation map</em> for region
512-
recolouring and changing the <em>gray-level histogram</em> for global luminance. Other
513-
descriptors remain unchanged, preserving geometry and non-target attributes.
539+
Because each descriptor governs a different attribute, editing is as simple as modifying
540+
that input and re-running the generator—no model fine-tuning or prompts required. Swapping
541+
the colour map alters regional colours while preserving shape and lighting; changing only
542+
the histogram shifts exposure/contrast without touching geometry or chroma. Qualitative and
543+
user-study results confirm higher naturalness, accuracy, and consistency than prompt-based
544+
baselines.
514545
</p>
515546

516547

517548
<h5 className="font-medium mb-2">1. Gray-Level Histogram Editing</h5>
549+
<p className="text-sm mb-3">
550+
We edit illumination by adjusting the grey-level histogram or applying histogram
551+
equalisation,
552+
then condition the generator with the updated histogram while keeping edges/colours fixed.
553+
</p>
518554
<div className="grid grid-cols-2 md:grid-cols-3 gap-3 mb-4">
519555
<figure className="rounded-xl border p-2">
520556
<img src={appAssets.editingEdgeHist} alt="Edge map"
@@ -532,9 +568,15 @@ export default function VisualSplitPage() {
532568
<figcaption className="mt-2 text-center text-xs">Ground Truth</figcaption>
533569
</figure>
534570
</div>
535-
<HistogramSlider before={appAssets.editingHistBefore} options={histogramOptions} />
571+
<HistogramSlider before={appAssets.editingHistBefore} options={histogramOptions}/>
536572

537573
<h5 className="font-medium mb-2">2. Colour Map Editing</h5>
574+
<p className="text-sm mb-3">
575+
To recolour specific regions, we modify only the segmented colour map (e.g., re-balanced hues or
576+
hand-edited segments). The diffusion model recolours those regions, respecting edge boundaries
577+
and the original illumination, and requires no text prompts—unlike baselines that need prompt
578+
engineering.
579+
</p>
538580
<div className="grid grid-cols-2 md:grid-cols-4 gap-3 mb-4">
539581
<figure className="rounded-xl border p-2">
540582
<img src={appAssets.editingEdgeColour} alt="Edge map"
@@ -587,7 +629,7 @@ export default function VisualSplitPage() {
587629

588630
{/* BibTeX */}
589631
<section id="bibtex" className="mb-12">
590-
<h3 className="text-2xl font-semibold mb-4">BibTeX</h3>
632+
<h3 className="text-2xl font-semibold mb-4">BibTeX</h3>
591633
<pre
592634
id="bibtex"
593635
className="bg-gray-50 p-3 rounded-lg overflow-x-auto text-xs"

project_page/components/OtherModelSlider.tsx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ export default function OtherModelSlider({ options, ours, gt }: OtherModelSlider
5151
alt="Ground truth"
5252
className="w-full object-cover rounded-lg"
5353
/>
54-
<figcaption className="mt-1 text-center text-xs">GT</figcaption>
54+
<figcaption className="mt-1 text-center text-xs">Original Image</figcaption>
5555
</figure>
5656
</div>
5757
<input

0 commit comments

Comments
 (0)