The idea of neural networks gets even more intriguing. We might ask: if we are learning those lower-layer logistic regressions — the different experts — what would be the target variable for each? While some practitioners build stacked models where the lower-layer experts are built to represent specific things using specific target variables (e.g., Perlich et al., 2013), more generally with neural networks target labels for training are provided only for the final layer (the actual target variable).
