Entropy and Zeta Functions for MCNN

Entropy and Zeta
Functions for MCNN
Jung-Chao Ban
Introduction
One-layer CNN
One-layer CNN
Without inputEntropy and Zeta Functions for
With input
Why sofic?One-dimensional Multi-layer Cellular
Sofic shift
Introduction to sofic shiftNeural Networks
Entropy for shift space
Zeta function
Application to one-layer
CNN with input
Difference between CNN∗
with and without inputJung-Chao Ban
Multi-layer CNN
Two-layer CNN with input
Multi-layer CNNDepartment of Mathematics,
National Hualien University of Education New phenomena
Recent works and
future plans
March, 2008
Reference
∗Joint work with Prof. Song-Sun Lin, Prof. Yin-Heng Lin and Mr. Chih-Hung
ChangEntropy and Zeta
Functions for MCNNOutline
Jung-Chao Ban
1 Introduction Introduction
2 One-layer CNNOne-layer CNN
One-layer CNN
One-layer CNN Without input
With input
Without input Why sofic?
With input Sofic shift
Introduction to sofic shiftWhy sofic?
Entropy for shift space
Zeta function
3 Sofic shift
Application to one-layer
CNN with input
Introduction to sofic shift Difference between CNN
with and without input
Entropy for shift space
Multi-layer CNN
Zeta function Two-layer CNN with input
Multi-layer CNN
Application to one-layer CNN with input
New phenomena
Difference between CNN with and without input
Recent works and
future plans
4 Multi-layer CNN
Reference
Two-layer CNN with input
Multi-layer CNN
5 New phenomena
6 Recent works and future plans
7 ReferenceEntropy and Zeta
Functions for MCNNOutline
Jung-Chao Ban
1 Introduction Introduction
2 One-layer CNNOne-layer CNN
One-layer CNN
One-layer CNN Without input
With input
Without input Why sofic?
With input Sofic shift
Introduction to sofic shiftWhy sofic?
Entropy for shift space
Zeta function
3 Sofic shift
Application to one-layer
CNN with input
Introduction to sofic shift Difference between CNN
with and without input
Entropy for shift space
Multi-layer CNN
Zeta function Two-layer CNN with input
Multi-layer CNN
Application to one-layer CNN with input
New phenomena
Difference between CNN with and without input
Recent works and
future plans
4 Multi-layer CNN
Reference
Two-layer CNN with input
Multi-layer CNN
5 New phenomena
6 Recent works and future plans
7 ReferenceEntropy and Zeta
Functions for MCNN
Jung-Chao Ban
L.O. Chua and L. Yang
Introduction
One-layer CNN
One-layer CNN
Without inputk =N
With input
Why sofic?
. . Sofic shift. .. . Introduction to sofic shift
Entropy for shift space
Zeta function
Application to one-layer
CNN with input
Difference between CNNk = 2
with and without input
Multi-layer CNN
Two-layer CNN with input
Multi-layer CNN
New phenomena
k = 1 Recent works and
future plans
Reference
(1)(1) {u }: input{u }: input ijℓ
1-d MCNN 2-d MCNNEntropy and Zeta
Functions for MCNNOne-dimensional Multi-layer CNN
Jung-Chao Ban
Introduction
One-layer CNN
One-layer CNN
Without input
(1)
With inputdx (1) (1) (1) (1) (1) (1)i
Why sofic?=−x + a y + b u +z ,i k i+k k i+kdt
Sofic shift|k|≤d |k|≤d
Introduction to sofic shift
(2)
Entropy for shift spacedx (2) (2) (2) (2) (2) (2)i
Zeta function=−x + a y + b u +z ,i k i+k k i+kdt Application to one-layer
CNN with input|k|≤d |k|≤d(1)
Difference between CNN
with and without input... Multi-layer CNN
Two-layer CNN with input
(N)
Multi-layer CNNdx (N) (N) (N) (N) (N) (N)i =−x + a y + b u +z ,i k i+k k i+k New phenomenadt
|k|≤d |k|≤d Recent works and
future plans
where Reference
(k) (k+1)
y = u , 1≤ k≤ N −1
l l
and
(1)
{u }: input data.l
>
X
X
X
>
>
>
>
>
>
>
>
>
X
>
8
>
>
>
>
>
>
<
X
>
>
>
>
>
>
>
>
>
>
>
>
>
:
XEntropy and Zeta
Functions for MCNN
Jung-Chao Ban
IntroductionConsiderN = 1 andd = 1,
One-layer CNN
One-layer CNNdxi
Without input=−x +a y +ay +a y +b u +bu +b u +z.i l i−1 i r i+1 l i−1 i r i+1
With inputdt
Why sofic?
Sofic shiftThe diagram of this CNN model:
Introduction to sofic shift
Entropy for shift space
Zeta function
Application to one-layer
CNN with input
Difference between CNNyk
with and without input
Multi-layer CNN
z yi Two-layer CNN with input
Multi-layer CNN
New phenomena
Recent works andi−1 ii+1
future plans
ReferenceukEntropy and Zeta
Functions for MCNN
Output function Jung-Chao Ban
Introduction1
y =f(x) = (|x+1|−|x−1|). One-layer CNN
2
One-layer CNN
Without input
y With input
Why sofic?
Sofic shift
y = f(x)
Introduction to sofic shift
1
Entropy for shift space
−1 Zeta function
x
1 Application to one-layer
CNN with input
1− Difference between CNN
with and without input
Multi-layer CNN
Two-layer CNN with input
Multi-layer CNN
New phenomena
Recent works and
future plans
ReferenceEntropy and Zeta
Functions for MCNN
Output function Jung-Chao Ban
Introduction1
y =f(x) = (|x+1|−|x−1|). One-layer CNN
2
One-layer CNN
Without input
y With input
Why sofic?
Sofic shift
y = f(x)
Introduction to sofic shift
1
Entropy for shift space
−1 Zeta function
x
1 Application to one-layer
CNN with input
1− Difference between CNN
with and without input
Multi-layer CNN
Two-layer CNN with input
Multi-layer CNN
Initial conditions New phenomena
Recent works and
(k) (k) (k) future plansu (0) =u andx (0) : constanti i i
ReferenceEntropy and Zeta
Functions for MCNN
Output function Jung-Chao Ban
Introduction1
y =f(x) = (|x+1|−|x−1|). One-layer CNN
2
One-layer CNN
Without input
y With input
Why sofic?
Sofic shift
y = f(x)
Introduction to sofic shift
1
Entropy for shift space
−1 Zeta function
x
1 Application to one-layer
CNN with input
1− Difference between CNN
with and without input
Multi-layer CNN
Two-layer CNN with input
Multi-layer CNN
Initial conditions New phenomena
Recent works and
(k) (k) (k) future plansu (0) =u andx (0) : constanti i i
Reference
ParametersEntropy and Zeta
Functions for MCNN
Output function Jung-Chao Ban
Introduction1
y =f(x) = (|x+1|−|x−1|). One-layer CNN
2
One-layer CNN
Without input
y With input
Why sofic?
Sofic shift
y = f(x)
Introduction to sofic shift
1
Entropy for shift space
−1 Zeta function
x
1 Application to one-layer
CNN with input
1− Difference between CNN
with and without input
Multi-layer CNN
Two-layer CNN with input
Multi-layer CNN
Initial conditions New phenomena
Recent works and
(k) (k) (k) future plansu (0) =u andx (0) : constanti i i
Reference
Parameters
(k)A : feedback template.