Optimization Hinge loss

plot of 3 variants of hinge loss function of z = ty: ordinary variant (blue), square (green), , piece-wise smooth version rennie , srebro (red).

however, since derivative of hinge loss @



t
y
=
1


{\displaystyle ty=1}

undefined, smoothed versions may preferred optimization, such rennie , srebro s

ℓ
(
y
)
=


{





1
2


−
t
y



if

 
 
t
y
≤
0
,






1
2


(
1
−
t
y

)

2





if

 
 
0
<
t
y
≤
1
,




0



if

 
 
1
≤
t
y








{\displaystyle \ell (y)={\begin{cases}{\frac {1}{2}}-ty&{\text{if}}~~ty\leq 0,\\{\frac {1}{2}}(1-ty)^{2}&{\text{if}}~~0<ty\leq 1,\\0&{\text{if}}~~1\leq ty\end{cases}}}

or quadratically smoothed

ℓ
(
y
)
=


1

2
γ



max
(
0
,
1
−
t
y

)

2




{\displaystyle \ell (y)={\frac {1}{2\gamma }}\max(0,1-ty)^{2}}

suggested zhang. modified huber loss special case of loss function



γ
=
2


{\displaystyle \gamma =2}

.