Questions tagged [perceptron]

Perceptron is a basic linear classifier that outputs binary labels.

Perceptron is a basic linear classifier that outputs binary labels. If the training data set is not linear separable, the learning algorithm cannot converge.

A classical problem of XOR is a dataset that is not linear separable. A perceptron does not work in this case. By adding nonlinear layers between the input and output, one can separate all data. With enough training data, the resulting network is able to model any well-defined function to arbitrary precision. This model is a generalization known as a multilayer perceptron.

For more details about perceptron, see wiki.

88 questions
14
votes
2 answers

Strange behavior with Adam optimizer when training for too long

I'm trying to train a single perceptron (1000 input units, 1 output, no hidden layers) on 64 randomly generated data points. I'm using Pytorch using the Adam optimizer: import torch from torch.autograd import Variable torch.manual_seed(545345) N,…
Bai Li
  • 243
  • 1
  • 2
  • 6
10
votes
3 answers

What is the difference between Perceptron and ADALINE?

What is the difference between Perceptron and ADALINE?
Developer
  • 1,069
  • 2
  • 9
  • 11
9
votes
4 answers

Why must a CNN have a fixed input size?

Right now I'm studying Convolutional Neural Networks. Why must a CNN have a fixed input size? I know that it is possible to overcome this problem (with fully convolutional neural networks etc...), and I also know that it is due to the fully…
9
votes
5 answers

Perceptron learning rate

Today I've seen many Perceptron implementations with learning rates. According to Wikipedia: there is no need for a learning rate in the perceptron algorithm. This is because multiplying the update by any constant simply rescales the weights…
Atte Juvonen
  • 323
  • 2
  • 5
  • 8
6
votes
1 answer

difference between empirical risk minimization and structural risk minimization?

I understand the meaning of empirical risk minimization as separate topic and was reading about structural risk minimization, it is hard for me to understand the difference between these two. I read somewhere that perceptron uses Emperical risk…
A.B
  • 316
  • 1
  • 3
  • 12
5
votes
4 answers

Is the prediction algorithm absolutely the same for all linear classifiers?

Is the prediction algorithm absolutely the same for all linear classifiers and linear regression algorithms? As known, any linear classifier can be described as: y = w1*x1 + w2*x2 + ... + c There are two broad classes of methods for determining the…
5
votes
1 answer

Whether add bias or not in a perceptron

In some places, perceptron is described as having added bias, while in some places, bias is not added. Which one is right for you?
Developer
  • 1,069
  • 2
  • 9
  • 11
5
votes
1 answer

Are single input single output neural networks possible?

This might be a weird question but I'm trying to have a deep understanding of how neural networks work theoretically. I was doing some tests with my perceptron and I decided to test it on a single input single output dataset. What I was looking for…
MaTHStudent
  • 151
  • 2
4
votes
1 answer

How to deal with a sparse matrix when using a perceptron based recommender system?

I'm constrained to use a perceptron based method. I have a user-item matrix filled with rating data on scale of 1 to 5 like this, with around 50% of the matrix with no data: r<- matrix(c(2,4, NA,5,NA,3,…
UD1989
  • 258
  • 1
  • 3
  • 6
4
votes
1 answer

What's wrong with my implementation of the Adaline algorithm?

I'm working through the textbook called Learning From Data and one of the problems from the first chapter has the reader implement the Adaline algorithm from scratch and I chose to do so using Python. The issue I'm running into is that the weights…
4
votes
2 answers

How do we define a linearly separable problem?

When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not. I am having trouble understanding this idea of linear…
Stefan Radonjic
  • 716
  • 1
  • 7
  • 20
4
votes
1 answer

Ambiguity in Perceptron loss function (C. Bishop vs F. Rosenblatt)

Bishop's Perceptron loss On one hand, it is stated in equation 4.54 of Chris Bishop's book (pattern recognition and machine learning) that the loss function of perceptron algorithm is given by: $${E_p}(\mathbf{w}) = - \sum\limits_{n \in M}…
pythinker
  • 1,237
  • 2
  • 7
  • 17
4
votes
1 answer

How to implement gradient descent for a tanh() activation function for a single layer perceptron?

I am required to implement a simple perceptron based neural network for an image classification task, with a binary output and a single layer, however I am having difficulties. I have a few problems: I am required to use a tanh() axtivation…
Waylander
  • 43
  • 4
4
votes
1 answer

Perceptron weight vector update

I read about the Rosenblatt Perceptron Learning Algorithm. Often there is an explicit note: It is important to note that all weights in the weight vector are being updated simultaneously But why are all weights updated simultaneously? I tried…
wake-0
  • 143
  • 5
3
votes
1 answer

What does n means in neural network neuron output?

I've found this equation that explains the output of a neuron in a MLP network: $y(n) = f(\mathbf{w}^T \mathbf{x}(n) + b)$ I can understand the general context, but since i have no background with mathematical notation, i don't understand what the…
heresthebuzz
  • 395
  • 3
  • 9
1
2 3 4 5 6