A direct variational approach to a problem arising in image reconstruction Luigi Ambrosio1 Simon Masnou2

Adirectvariationalapproachtoaproblemarisinginimage
reconstruction
LuigiAmbrosio
1
SimonMasnou
2

Abstract
Weconsideravariationalapproachtotheproblemofrecoveringmissingpartsina
panchromaticdigitalimage.Representingtheimagebyascalarfunction
u
,wepropose
amodelbasedontherelaxationoftheenergy
ur|r
u
|
(

+

|
div
|
p
)
,,>
0
,p

1
Z|ur|whichtakesintoaccounttheperimeterofthelevelsetsof
u
aswellastheL
p
norm
ofthemeancurvaturealongtheirboundaries.Weinvestigatethepropertiesofthis
variationalmodelandtheexistenceofminimizingfunctionsinBV.Wealsoaddress
relatedissuesforintegralvarifoldswithgeneralizedmeancurvatureinL
p
.
Keywords:
Imageprocessing;imagereconstruction;BV;meancurvature;varifolds;
relaxation.

1Introduction
Manyproblemsindigitalimageprocessingrequiretheabilitytorecovermissingpartsof
animageortoremovespuriousorundesiredobjects.Onecanmentionforinstancethe
removalofscratchesinoldphotographsandlms,therecoveryofpixelsblockscorrupted
duringabinarytransmission(oranalogouslytheremovalofimpulsenoise)ortheremoval
ofundesiredpublicity,textorsubtitlesfromaphotograph.Onecanalsothinktospecial
eectsformoviepostproduction,e.g.theremovalofamicrophoneappearinginascene.
Adigitalimageisusuallymodeledasafunction
u
fromaboundeddomainofIR
N
(
N
=2
forusualsnapshots,
N
=3formedicalimagesormovies,
N
=4formovingmedicalimages)
ontoIR
M
(
M
=1foragrey-levelimage,
M
=3forcolourimages).Sinceitisnowwell
1
ScuolaNormaleSuperiore,PiazzadeiCavalieri7,56126Pisa,Italy,luigi@ambrosio.sns.it
2
Lab.J.-L.Lions,B.C.187,Univ.PierreetMarieCurie,75252ParisCedex05,France,mas-
nou@ann.jussieu.fr

1

admittedthattheessentialfeaturesofanynaturalimagearecontainedinitsgreylevel
representation,weshallconcentrateonthepanchromaticcase
M
=1.Toextendtothe
colourcaseanoperatordesignedforgreylevelimages,itisgenerallyenoughtoprocess
separatelyeachchannelinthecolourrepresentation,e.g.thered-green-bluerepresentation
or,moreappropriately,anyrepresentationwithtwochannelsforthechromaticityandone
channelfortheluminosity(see[9]andthereferencesherein).
AftertheworkofL.RudinandS.Osher[34],theusualrepresentationofapanchromatic
imageisasumoftwocomponents
u
1
∈
BV(IR
N
)and
u
2
∈
L
2
(IR
N
).Thecomponent
u
1
issupposedtodescribethe
geometry
oftheimage,i.e.itsobjectsandtheirboundaries,
while
u
2
containsallinformationabout
texture
and
additivenoise
.Theassumptionthat
thegeometryoftheimagecanbedescribedbyafunctionofboundedvariationsounds
quitenatural,foritmeansthattherecanbediscontinuitiesintheimagebutsupportedon
rectiablecurves.Thenecessityofanothercomponentthatdoesnotnecessarilybelongs
toBVcanbecorroboratedbyanexperimentalprocedurethatseemstoindicatethat,
givenadigitalimage,thesubjacent“real”imagemaybeoftentoooscillatingtobelongto
BV(see[2]forthedetailsand[11]forconnectedtheoreticissues).Thereadermayrefer
to[4,20]foradetailedsurveyofthespaceBV.
Amongthelargeliteraturethathasbeenpublishedinrecentyearsontherecovery
ofmissingpartsinadigitalimage,onecanbasicallydistinguishbetweentwoapproaches
andeachofthemcorrespondsinsomewaytotheprocessingofonecomponentinthe
decompositionabove:
thestochasticapproach,whichisbasedonthemodelingofanimageasarealization
ofarandomprocess.Usually,itisassumedthattheimageintensityderivesfroma
MarkovRandomFieldand,therefore,satisespropertiesoflocalityandstationarity,
i.e.eachpixelisonlyrelatedtoasmallsetofneighboringpixelsanddierentregions
oftheimageareperceivedsimilar.Thismodelingisparticularlyadaptedfortexture
images(thustotheprocessingorthecomponent
u
2
inthepreviousdecomposition)
andhasmotivatednumerousworksontextureanalysisandsynthesis[5,14,15,25,
32,33,42,44],
thedeterministicapproach,whosemainpurposeistorecoverthegeometryofthe
image.Themodelweshalldiscussinthispaperbelongstothiscategory.
ApioneeringworkontherecoveryofplaneimagegeometryisduetoD.Mumford,
M.NitzbergandT.Shiota[31].Theydidnotdirectlyaddresstheproblemofrecovering
missingpartsinanimagebutrathertriedtoidentifyoccludingandoccludedobjectsin
ordertocomputetheimagedepthmap.Theiralgorithmstartswiththedetectionofthe
boundariesofimageobjects.Thenextstepistheidenticationofoccludedandoccluding
objects.Tothisaim,Nitzberg,MumfordandShiotahadtheluminousideatomimic
anaturalabilityofhumanvisiontocompletepartiallyoccludedobjects,theso-called

2

amodalcompletion
processdescribedandstudiedbytheGestaltschoolofpsychologyand
particularlyG.Kanizsa[23].Fromaseriesofperceptualexperiments,Kanizsafoundout
thatourvisionsystemdetectsocclusionataverylowlevel,actuallyassoonasitdetects
T-junctions
,whicharepointswhereanobjectoutlineabruptlyabutsagainsttheoutline
ofanotherobjectandformsajunctionintheshapeoftheletter“T”.Inparticular,our
perceptionofocclusionhasnothingtodowithapriorrecognitionoftheobjects.Being
theT-junctiondetected,ourbrainperformsacontinuationofobjectsboundariesbetween
T-junctions(seegure1).

T−junctions

Figure1:Thisexample,duetoG.Kanizsa[23],illustratestheamodalcompletionprocess.
Startingfromthefourobjectsontheleftcolumn,theadditionofeitherfourwhiterectangles
orawhitecrossproducesT-junctions(middlecolumn),thatconduceourbraintoperceive
occlusionsthat,inreality,donotexist.Thisillustratesperfectlythelinkbetweenthe
presenceofT-junctionsandtheperceptionofocclusions.Then,ourvisualsystemrecovers
thevirtuallyoccludedobjects(fourblackdisksinonecaseandablacksquareintheother)
byconnectingT-junctionswithcompletioncurves,followinga
goodcontinuation
principle.
Wehaverepresentedthosecurveswithdashlinesontherightcolumn.

AspointedoutbyKanizsa,thiscontinuationprocessreliesonmanydierentlaws[23]
andthereisactuallynoobviousmaytomodelit,eveninrelativelysimplesituations[18].
Again,itseemsthatnoprocessofrecognitionbeinvolved(see