$Learning from Demonstration for Manufacturing Tasks: A Workpiece-Geometry-First Pipeline Trained on YouTube Video Demonstrations$

$This paper presents a Learning from Demonstration (LfD) structured six-stage process designed to train a Machine Learning (ML) agent to perform metal forming tasks by observing YouTube video demonstrations of skilled blacksmiths. The research objective was to apply imitation learning methods from ML to the problem of automating metal forming knowledge in manufacturing engineering. This is an underexplored domain where prior work has relied almost exclusively on controlled laboratory settings with instrumented tools and calibrated sensors (Argall et al., 2009; Niekum et al., 2015), making it inaccessible for small-scale craft manufacturing contexts. This research addressed a fundamental challenge that distinguishes it from prior video-based LfD work. First, YouTube videos were not designed as training data. Second, conventional approaches that track the human operator's (blacksmith) hands fail as manufacturing video is recorded to show the work rather than the worker. To address the problem, this research presents a workpiece-geometry-first extraction approach that derives state-action pairs. The derivation was done entirely from how the metal workpiece changes shape between consecutive video frames. Behavioral Cloning (BC) was selected as the core method because it enabled imitation learning by training the model to replicate demonstrated behaviors through supervised learning. The steps for BC involved mapping observed workpiece states directly to the human operator's actions. In addition, a two-layer Long Short-Term Memory (LSTM) policy network was trained using BC on the extracted video frames. LSTM was specifically chosen over a standard feedforward network to capture the history-dependent nature (Mandlekar et al., 2021; Rahmatizadeh et al., 2018) of forming tasks where the correct action depends on what has already been done to the workpiece. The next process incorporated a 2-dimension deformable polygon simulator for trained policy evaluation using K-means clustering-based waypoint planner. Furthermore, a Reinforcement Learning (RL) agent was added to learn through trial and error by receiving reward signals from the environment. This preference-based reward model was trained on human expert comparisons to capture quality judgments that geometric metrics alone cannot express. Results showed that the LSTM policy converged with a Mean Squared Error (MSE) loss of 0.010 by epoch 98, which was identified as the best checkpoint with an overall shape dimension. On the other hand, LSTM struggled most with predicting the precise contact point, with a Mean Absolute Error (MAE) of 0.087. Despite this struggle, the waypoint planner was able to produce coherent intermediate shape sequences from noisy demonstration data. This indicates that meaningful forming stages can still be identified from uncontrolled video without manual annotation. Lastly, the paper acknowledges the ethical implications of this approach. For instance, extracting and automating metal forming knowledge from publicly available YouTube videos raises questions of consent, attribution, and the preservation of skilled blacksmith communities. The paper thus presents both the promise and the current limitations of using uncontrolled online video as a source of metal forming demonstration data.$