Machine Learning
84 curated documents on machine learning from the Gyre Research library, each with a summary. Free to read, no signup required.
A Few Useful Things to Know about Machine Learning
Pedro Domingos · Paper
Seminal paper by Pedro Domingos (2012). Access: open-access. Source: https://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf
A Neural Network Approach to Understanding Implied Volatility Movements
Jay Cao, Jacky Chen & John Hull · Report
Abstract We employ neural networks to understand volatility surface movements. We first use daily data on options on the S&P 500 index to derive a relationship between the expected change in implied volatility and three variables: the return on the index, the moneyness of the option, and the remaining life of the option. This model provides an improvement of 10.72% compared with a simpler analytic model. We then enhance the model with an additional feature: the level of the VIX index prior to the change being observed.
Also filed under Quant Finance, Risk Management
Adam: A Method for Stochastic Optimization
Diederik P. Kingma and Jimmy Ba · Report
The 2015 paper introducing the Adam optimizer, combining adaptive per-parameter learning rates with momentum. Became the default optimizer for deep learning and remains the standard choice for training large models.
Adam: A Method for Stochastic Optimization
Kingma and Ba · Paper
Seminal paper by Kingma and Ba (2014). Access: open-access. Source: https://arxiv.org/abs/1412.6980
Advanced Data Analytics Using Python
Sayan Mukhopadhyay · Book
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this ...
Also filed under Python, Data Visualization
Advancing into Analytics
George Mount · Book
Foundations of Analytics in Excel 1. Foundations of Exploratory Data Analysis. 3 What Is Exploratory Data Analysis?
Also filed under Data Visualization, SQL
Agile Machine Learning: Effective Machine Learning Inspired by the Agile Manifesto
Eric Carter & Matthew Hurst · Book
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this ...
Algorithms Python
Yang Hu · Notes
Algorithms Python YANG HU Simple is the beginning of wisdom. From the essence of practice, this book to briefly explain the concept, and vividly cultivate programming interest , you will learn it easy fast and well.
Also filed under Python, Computer Science
An Introduction To Statistical Learning with Applications in Python
Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, Jonathan Taylor · Book
"An Introduction to Statistical Learning with Applications in Python" is an essential guide to statistical and machine learning methods, providing a clear and accessible introduction to the field. Designed for students, researchers, and practitioners, this book offers a balance between theory and practical implementation, focusing on Python-based applications for data analysis and predictive modeling. Key Highlights: Comprehensive Coverage of Statistical Learning: The book explains core machine learning principles, including supervised and unsupervised learning, model evaluation, and statistical inference. Regression and Classification Techniques: Topics include linear regression, logistic regression, decision trees, and support vector machines (SVMs), providing practical guidance on applying these models effectively. Resampling Methods and Model Selection: Introduces cross-validation and bootstrap techniques, helping readers understand how to improve model performance and avoid overfitting. Tree-Based and Ensemble Methods: Covers random forests, gradient boosting, and bagging, which are widely used in modern machine learning applications. Unsupervised Learning Techniques: Introduces principal component analysis (PCA), k-means clustering, and hierarchical clustering, essential for working with high-dimensional data. Python-Based Implementation: The book provides hands-on coding examples using scikit-learn, NumPy, pandas, and Matplotlib, enabling readers to apply theoretical concepts in real-world scenarios. This book serves as a valuable resource for those looking to build a strong foundation in statistical learning, blending intuitive explanations, practical examples, and Python-based applications to make complex concepts accessible to a broad audience.
Also filed under Python, Statistics
Applied Machine Learning with Python
Andrea Giussani · Book
Via Salasco, 5 - 20136 Milano Tel.
Artificial Intelligence And Machine Learning In Financial Services: Opportunities And Challenges In Anti-Money Laundering And Combatting The Financing Of Terrorism
The Association Of The Bar Of The City Of New York · Report
Introduction ........................................................................................................................ What Are the Definitions of Artificial Intelligence and Machine Learning? What Are Some of the Use Cases for AI/ML in the Financial Services Sector?............ What Are Some of the Risks Associated With AI/ML?
Also filed under Quant Finance, Finance
Artificial Intelligence Index Report 2024
Ray Perrault & Jack Clark · Book
The "Artificial Intelligence Index Report 2024" is the seventh edition of an annual comprehensive analysis that tracks, collates, distills, and visualizes data related to artificial intelligence (AI). Produced by the Stanford Institute for Human-Centered Artificial Intelligence (HAI), this report aims to provide unbiased, rigorously vetted, and broadly sourced data to inform policymakers, researchers, executives, journalists, and the general public about the complex field of AI. Key Highlights: Expansion of Scope: This edition broadens its coverage to include essential trends such as technical advancements in AI, public perceptions of the technology, and the geopolitical dynamics surrounding its development. Original Data and New Analyses: Featuring more original data than previous editions, the report introduces new estimates on AI training costs, detailed analyses of the responsible AI landscape, and an entirely new chapter dedicated to AI's impact on science and medicine. Generative AI Investment Surge: Despite a decline in overall AI private investment in 2023, funding for generative AI surged, nearly octupling from 2022 to reach $25.2 billion. Major players in the generative AI space, including OpenAI, Anthropic, Hugging Face, and Inflection, reported substantial fundraising rounds. U.S. Leadership in AI Investment: The United States saw AI investments reach $67.2 billion in 2023, nearly 8.7 times more than China, the next highest country. Impact on Science and Medicine: An entirely new chapter is dedicated to AI's impact on science and medicine, reflecting the growing influence of AI technologies in these critical fields. This report serves as an essential resource for understanding the current state and future trajectory of AI, offering a blend of data-driven insights and comprehensive analyses across various dimensions of artificial intelligence.
Also filed under AI, Statistics
Artificial Intelligence with Python
Tutorials Point · Book
About the Tutorial Artificial intelligence is the intelligence demonstrated by machines, in contrast to the intelligence displayed by humans. This tutorial covers the basic concepts of various fields of artificial intelligence like Artificial Neural Networks, Natural Language Processing, Machine Learning, Deep Learning, Genetic algorithms etc., and its implementation in Python. Audience This tutorial will be useful for graduates, post graduates, and research students who either have an interest in this subject or have this subject as a part of their curriculum. The reader can be a beginner ...
Artificial Intelligence/ Machine Learning Explained
Steve Blank · Guide
about the competitive edge they’d have by today in business or as a nation. That’s where we are today with Artificial Intelligence and Machine Learning. These technologies will transform businesses and government agencies. Today, 100s of billions of dollars in private capital have been invested in 1,000s of AI startups.
Also filed under AI, Statistics
Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin (Google Brain) · Report
The 2017 NeurIPS paper that introduced the Transformer architecture — the foundation of every modern large language model. Dispensing with recurrence and convolutions entirely, the Transformer uses multi-head self-attention to parallelize sequence modeling. The single most-cited deep learning paper of the last decade.
Also filed under AI, Computer Science
Attention Is All You Need
Vaswani et al. · Paper
Seminal paper by Vaswani et al. (2017). Access: open-access. Source: https://arxiv.org/abs/1706.03762
Automated Machine Learning: Methods, Systems, Challenges
Frank Hutter, Lars Kotthoff, Joaquin Vanschoren (Editors) · Book
An edited volume in The Springer Series on Challenges in Machine Learning, covering methods, systems, and open challenges in automated machine learning (AutoML). Contributions span algorithmic foundations, hyperparameter optimization, neural architecture search, and real-world applications.
Also filed under AI, Computer Science
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe and Christian Szegedy (Google) · Report
The 2015 paper introducing batch normalization — a technique that normalizes layer inputs during training, dramatically accelerating convergence and enabling higher learning rates. Standard component of modern CNN and Transformer architectures.
Also filed under AI, Statistics
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova (Google AI Language) · Report
The 2018 paper introducing BERT, which established the pre-train/fine-tune paradigm that dominated NLP before the GPT era. Masked language modeling + next-sentence prediction on unlabeled text produced representations transferable to a wide range of downstream tasks.
Also filed under AI, Computer Science
Big Data and Artificial Intelligence in Digital Finance
John Soldatos & Dimosthenis Kyriazis · Book
Preface The finance sector is among the most data-savvy and data-intensive of the global economy. The ongoing digital transformation of financial organizations, along with their interconnection as part of a global digital finance ecosystem, is producing petabytes of structured and unstructured data. The latter represent a significant opportunity for banks, financial institutions, and financial technology firms (Fin- Techs): Leveraging these data financial organizations can significantly improve both their business processes and the quality of their decisions. As a prominent example, modern banks can...
Also filed under AI, Quant Finance
Building Machine Learning Systems with a Feature Store
Jim Dowling · Book
O'Reilly Media early-release (2025) on designing and operating ML systems built around a feature store. Covers feature engineering pipelines, online/offline serving, data consistency, and the architecture of modern ML platforms.
Also filed under Software Engineering, AI
Building Machine Learning Systems with Python
Book
A practical introduction to machine learning using Python, covering NumPy, SciPy, and Matplotlib for data handling; classification (kNN, naive Bayes), clustering, and feature engineering; hands-on examples on the Iris and Seeds datasets; techniques for evaluation and model selection.
Also filed under Python, Computer Science
Constitutional AI: Harmlessness from AI Feedback
Yuntao Bai et al. (Anthropic) · Report
The 2022 Constitutional AI paper introducing RLAIF — reinforcement learning from AI feedback — as a scalable alternative to RLHF. Models critique and revise their own outputs against a set of written principles. The foundation of Claude's training methodology.
Also filed under AI
CS229 Lecture Notes
Andrew Ng & Tengyu Ma · Report
The CS229 Lecture Notes are comprehensive materials from Stanford University's Machine Learning course, CS229, primarily authored by Andrew Ng and later updated by Tengyu Ma. These notes serve as a foundational resource for understanding various machine learning algorithms and principles. Key Highlights: Supervised Learning: Linear Regression: Introduces the Least Mean Squares (LMS) algorithm and the normal equations for parameter estimation. Logistic Regression: Discusses classification tasks and the application of logistic functions. Regularization: Explores techniques like L1 and L2 regularization to prevent overfitting. Unsupervised Learning: Clustering: Covers methods such as the k-means algorithm for grouping similar data points. Dimensionality Reduction: Explains Principal Component Analysis (PCA) for reducing feature space while preserving variance. Reinforcement Learning: Markov Decision Processes (MDPs): Provides a framework for modeling decision-making in environments with uncertainty. Policy Gradient Methods: Discusses approaches for optimizing policies in reinforcement learning scenarios. These lecture notes are available on the official CS229 website and have been utilized globally by students and professionals seeking a structured understanding of machine learning concepts. For more detailed information, you can refer to the CS229 Lecture Notes.
Also filed under Computer Science, Math
Deep Learning for Computer Vision with Python
Adrian Rosebrock · Book
Deep Learning for Computer Vision with Python Practitioner Bundle Dr. Books like this are made possible by the time invested by the authors. First printing, September 2017 To my father, Joe; my wife, Trisha; and the family beagles, Josie and Jemma. Without their constant love and support, this book would not be possible.
Also filed under Python, Computer Science
Deep Learning with Azure
Mathew Salvaris, Danielle Dean, Wee Hyong Tok · Book
"Deep Learning with Azure" by Mathew Salvaris, Danielle Dean, and Wee Hyong Tok is a practical guide for data scientists, engineers, and AI practitioners who want to harness the power of Microsoft Azure to build, train, and deploy deep learning models. Through a series of hands-on tutorials, the book covers everything from setting up the development environment and creating deep learning models to deploying those models at scale on Azure. Key Concepts Covered Introduction to Azure and Deep Learning: The book begins with an introduction to Azure, particularly Azure Machine Learning (AML), a cloud-based service that simplifies the deployment of machine learning models. It explains how to use Azure's services to handle the computational resources required for deep learning tasks. Setting Up Your Deep Learning Environment: A key focus of the book is the setup and configuration of the deep learning environment on Azure. The authors walk through the steps required to set up Azure Machine Learning workspaces, create compute clusters, and use Azure Notebooks for efficient experimentation. The authors also discuss the integration of Azure Databricks for big data analytics and model training. Building Deep Learning Models: The book provides practical, hands-on examples of building various types of deep learning models, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative models, using frameworks like TensorFlow, Keras, and PyTorch on Azure. By the end of the book, readers will have the skills necessary to leverage Azure’s cloud capabilities for deep learning, optimizing workflows, scaling model training, and deploying models for real-world applications. The book provides a comprehensive and structured approach to using Azure for deep learning, making it an essential resource for anyone working with AI in the cloud.
Deep Learning with PyTorch
Eli Stevens, Luca Antiga, Thomas Viehmann · Book
"Deep Learning with PyTorch" by Eli Stevens, Luca Antiga, and Thomas Viehmann is an insightful and practical guide to mastering deep learning using the PyTorch framework. The book is designed for those who are familiar with basic machine learning concepts and want to dive deeper into the world of deep learning with a hands-on approach. Key Concepts Covered: Introduction to PyTorch: The book begins with an introduction to the PyTorch framework, highlighting its flexibility, ease of use, and efficient handling of dynamic neural networks. It emphasizes how PyTorch’s tensor operations form the backbone of deep learning models, similar to NumPy but with added support for GPU acceleration. Tensors and Autograd: The authors provide an in-depth exploration of tensors, the fundamental data structure in PyTorch, explaining their creation, manipulation, and efficient use. They also introduce autograd, PyTorch's automatic differentiation library, which simplifies the process of computing gradients for backpropagation in neural networks. Building Neural Networks: The book walks through the steps of constructing various neural network architectures using PyTorch's nn.Module class. It covers feedforward neural networks, convolutional neural networks (CNNs), and recurrent neural networks (RNNs), providing code examples and explanations on how to define, train, and evaluate these models. Training Neural Networks: A significant portion of the book focuses on model training, including defining loss functions, choosing optimizers (like SGD and Adam), and managing training loops. The authors dive into best practices for handling overfitting, using techniques like dropout, batch normalization, and data augmentation. Working with Real-World Data: The book emphasizes how to preprocess and load real-world data for deep learning tasks, including using PyTorch's DataLoader and Dataset classes. It also discusses the importance of data preprocessing and augmentation for improving model generalization. Transfer Learning: One of the standout sections of the book is on transfer learning. It explains how to leverage pre-trained models (like ResNet, VGG, etc.) for tasks like image classification, speeding up the training process and improving accuracy, especially when data is scarce. Generative Models and Advanced Topics: The book introduces advanced deep learning topics such as Generative Adversarial Networks (GANs), reinforcement learning, and other generative models, helping readers expand their knowledge into more complex areas of deep learning. Practical Considerations: The authors also provide practical insights on model evaluation, deployment, and debugging. They explain how to tune hyperparameters effectively and optimize models for deployment on CPUs and GPUs. Summary: "Deep Learning with PyTorch" is a comprehensive, practical guide that equips readers with the knowledge and tools to build state-of-the-art deep learning models using PyTorch. The book offers clear explanations, hands-on examples, and expert advice, making it an invaluable resource for both beginners and those looking to deepen their understanding of deep learning techniques. By the end of the book, readers will have a solid understanding of PyTorch's core concepts and be able to implement and train various types of deep learning models for real-world applications.
Also filed under Math, Computer Science
Deep Neural Networks and Data for Automated Driving
Tim Fingscheidt, Hanno Gottschalk & Sebastian Houben · Book
This book is an open access publication. The images or other third party material in this book are included in the book’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. The use of general descriptive names, registered names, trademarks, service marks, etc.
Also filed under AI, Computer Science
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun (Microsoft Research) · Report
The 2015 paper introducing residual connections and ResNet — the architecture that enabled training of networks 100+ layers deep. Residual connections solved the degradation problem in deep networks and became a near-universal building block of modern deep learning.
Also filed under AI, Computer Science
Deep Residual Learning for Image Recognition
He et al. · Paper
Seminal paper by He et al. (2015). Access: open-access. Source: https://arxiv.org/abs/1512.03385
Dive into Deep Learning
Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola · Book
"Dive into Deep Learning" is an open-source book that provides a comprehensive, hands-on introduction to deep learning techniques. Written by Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola, the book offers an in-depth exploration of the fundamental principles and applications of deep learning, with a focus on practical implementation using modern deep learning frameworks like Apache MXNet and PyTorch. Key Highlights: Introduction to Deep Learning: The book starts with an introduction to deep learning, explaining its significance and how it differs from traditional machine learning techniques. It covers the historical context of neural networks and their evolution into deep learning, highlighting key breakthroughs such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Mathematical Foundations: The authors provide a solid mathematical foundation for understanding deep learning. Topics like linear algebra, probability, optimization, and calculus are discussed, as they form the backbone of many deep learning algorithms. This foundational knowledge is essential for readers to grasp how deep learning models work under the hood. Neural Networks and Backpropagation: The book thoroughly explains the architecture of neural networks, covering perceptrons, multi-layer perceptrons (MLPs), and how backpropagation works to update model weights during training. The authors emphasize understanding how the model learns and how to compute gradients using the chain rule to minimize the loss function. Training Deep Networks: In this section, the book delves into the critical aspects of training deep learning models, including optimization algorithms (such as gradient descent), weight initialization, regularization techniques (dropout, batch normalization), and activation functions. The authors also discuss challenges in training deep models, such as vanishing and exploding gradients. Convolutional Neural Networks (CNNs): CNNs are covered in depth, as they are essential for tasks like image recognition and computer vision. The book explains convolution layers, pooling layers, and how CNNs can be used to automatically learn hierarchical feature representations from raw input data. It also provides hands-on examples for building and training CNNs for image classification tasks. Recurrent Neural Networks (RNNs) and Sequence Models: The book introduces RNNs and their variants, such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), which are used for sequential data. It explores how RNNs can model time-dependent processes, such as speech recognition, language modeling, and machine translation. Generative Models: The authors introduce generative models, particularly Generative Adversarial Networks (GANs), which are used to generate new data samples that resemble a given dataset. GANs are explored with practical examples, showing their potential in generating images, videos, and even music. Transfer Learning: Transfer learning is another key topic in the book, where pre-trained models are fine-tuned for new tasks. The authors show how leveraging models that have been trained on large datasets can help with tasks where data is limited, improving model performance and reducing training time. Advanced Topics: The book also touches on more advanced topics in deep learning, such as reinforcement learning, attention mechanisms, and transformers, which have been key in the development of state-of-the-art models for tasks like natural language processing (NLP) and image generation.
Also filed under Math, Computer Science
Dropout: A Simple Way to Prevent Neural Networks from Overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov (University of Toronto) · Report
The 2014 JMLR paper introducing dropout as a regularization technique. Randomly dropping units during training prevents co-adaptation and acts as an approximate ensemble method. A foundational technique used in nearly every deep neural network.
Also filed under AI, Statistics
Efficient Exploration for LLMs
Vikranth Dwaracherla, Seyed Mohammad Asghari, Botao Hao, Benjamin Van Roy · Report
"Efficient Exploration for LLMs" is a research paper authored by Vikranth Dwaracherla, Seyed Mohammad Asghari, Botao Hao, and Benjamin Van Roy, focusing on enhancing large language models (LLMs) through efficient exploration strategies in gathering human feedback. Key Contributions: Efficient Query Generation: The study introduces an agent that sequentially generates queries while concurrently fitting a reward model based on the feedback received. This approach aims to optimize the learning process by selecting the most informative queries. Double Thompson Sampling: The researchers employ double Thompson sampling for query generation, utilizing epistemic neural networks to represent uncertainty. This method balances exploration and exploitation, leading to more effective learning with fewer queries. Performance Improvement: The findings demonstrate that efficient exploration enables high levels of performance with significantly fewer queries, highlighting the importance of uncertainty estimation and the choice of exploration strategy in training LLMs. This research underscores the potential of incorporating advanced exploration techniques to enhance the efficiency and effectiveness of large language models, particularly in the context of human-in-the-loop training scenarios.
Also filed under AI, Quant Finance
Exercises in Machine Learning
Michael U. Gutmann · Book
"Exercises in Machine Learning" by Michael U. Gutmann is a comprehensive and hands-on resource for anyone seeking to deepen their understanding of machine learning algorithms through practical exercises. Designed for students, practitioners, and machine learning enthusiasts, the book emphasizes applying theoretical knowledge to real-world problems, ensuring readers not only understand key concepts but also know how to implement them effectively. Key Highlights: Comprehensive Coverage of Machine Learning Topics: This book covers a broad range of machine learning techniques, offering exercises on both supervised and unsupervised learning, model evaluation, and optimization. It helps readers understand the core principles of machine learning and how to implement them practically. Supervised Learning Techniques: Key algorithms such as linear regression, logistic regression, support vector machines (SVM), and decision trees are explored in detail. Each exercise provides hands-on experience, allowing readers to apply these models effectively to real datasets, making complex topics more accessible. Unsupervised Learning Methods: Unsupervised techniques like k-means clustering, hierarchical clustering, and principal component analysis (PCA) are covered with practical exercises to help readers work with unlabelled data and uncover hidden patterns. Model Evaluation and Tuning: The book introduces essential model evaluation methods such as cross-validation, bias-variance tradeoff, and overfitting, helping readers understand how to assess model performance. Additionally, it covers hyperparameter tuning, including techniques like grid search and random search to enhance model accuracy. Advanced Topics: For those looking to explore more advanced machine learning techniques, the book touches on deep learning, reinforcement learning, and Bayesian methods, providing exercises that introduce these complex methods in a manageable and practical way. Python-Based Implementation: Each exercise is designed with Python in mind, using libraries such as NumPy, scikit-learn, and TensorFlow. This allows readers to apply machine learning concepts using the same tools that data scientists and machine learning engineers use in the industry.
Also filed under Math, Computer Science
Exploring Synthetic Data for Artificial Intelligence and Autonomous Systems: A Primer
Harry Deng · Report
UNIDIR (United Nations Institute for Disarmament Research) primer on synthetic data: how it is generated, why it is used in AI and autonomous systems, and the security and policy implications for militarized applications. Published 2023 under UNIDIR Security and Technology Programme.
Also filed under AI
Financial Statement Analysis with Large Language Models
Alex G. Kim, Maximilian Muhn & Valeri V. Nikolaev · Report
Abstract We investigate whether an LLM can successfully perform financial statement analy- sis in a way similar to a professional human analyst. We provide standardized and anonymous financial statements to GPT4 and instruct the model to analyze them to determine the direction of future earnings. Even without any narrative or industry- specific information, the LLM outperforms financial analysts in its ability to predict earnings changes. The LLM exhibits a relative advantage over human analysts in sit- uations when the analysts tend to struggle.
Also filed under Risk Management, Corporate Finance
Flexible Automation and Intelligent Manufacturing: The Human-Data- Technology Nexus
Kyoung-Yun Kim, Leslie Monplaisir & Jeremy Rickli · Book
This book is an open access publication, corrected publication 2023. The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book'
Foundation Models for Natural Language Processing
Gerhard Paaß & Sven Giesselbach · Book
It brings together the latest developments in all areas of this multidisciplinary topic, ranging from theories and algorithms to various important applications. Furthermore, it supports Open Access publica- tion mode. This book is an open access publication. The images or other third party material in this book are included in the book’s Creative Commons license, unless indicated otherwise in a credit line to the material.
Also filed under AI, Software Engineering
Foundations of Machine learning - Lecture Notes
Ajay Nagesh · Report
"Foundations of Machine Learning - Lecture Notes" by Ajay Nagesh is an in-depth guide to the foundational concepts of machine learning, offering a comprehensive overview of the theoretical underpinnings and practical applications of the field. The book is designed for students, professionals, and researchers who seek to understand the core principles of machine learning and their real-world applications. Key Highlights: Introduction to Machine Learning Concepts: The book starts with a strong introduction to machine learning, covering its history, evolution, and various paradigms such as supervised learning, unsupervised learning, and reinforcement learning. It establishes a solid theoretical foundation for understanding how machines can learn from data. Mathematical Foundations: The book dives into the key mathematical concepts that form the backbone of machine learning, such as linear algebra, probability theory, optimization, and statistics. These mathematical tools are essential for building, understanding, and analyzing machine learning models. Supervised Learning: A significant portion of the book focuses on supervised learning algorithms. The text explains the mechanics and applications of popular models, such as linear regression, logistic regression, decision trees, and support vector machines (SVMs). Each model is accompanied by examples, explaining the mathematical formulation, training process, and how to evaluate model performance. Unsupervised Learning: The book covers unsupervised learning techniques like clustering, dimensionality reduction, and anomaly detection. It introduces methods such as k-means clustering, hierarchical clustering, principal component analysis (PCA), and t-SNE, providing a solid understanding of how these techniques can be applied to real-world data without labeled outputs. Optimization and Model Evaluation: A critical section of the book is dedicated to optimization techniques used in machine learning, including gradient descent, stochastic gradient descent, and regularization methods. It also explores various performance metrics like accuracy, precision, recall, F1-score, and cross-validation, offering insights into how to evaluate model performance effectively. Deep Learning Basics: While the book focuses on foundational machine learning techniques, it also introduces readers to deep learning, discussing neural networks, backpropagation, and architectures like convolutional neural networks (CNNs) and recurrent neural networks (RNNs). This section serves as a bridge between classical machine learning and advanced deep learning techniques. Practical Applications: The book is rich with real-world examples and case studies, demonstrating how machine learning concepts can be applied to a wide range of fields, such as healthcare, finance, natural language processing (NLP), and computer vision. It highlights how machine learning models can be built and deployed to solve practical problems in these domains. Ethical Considerations: In addition to technical content, the book touches upon the ethical implications of machine learning, such as bias in data, fairness in algorithms, and the responsible use of AI. It discusses how to build ethical models and the importance of transparency in machine learning systems. Hands-On Learning: The book encourages hands-on learning by providing coding exercises and examples in Python, using libraries such as scikit-learn, TensorFlow, and Keras. The practical approach allows readers to implement and test the models they learn about in the book, reinforcing the theoretical concepts. Future Trends in Machine Learning: The book concludes with a discussion on emerging trends in machine learning, such as transfer learning, explainable AI (XAI), and the integration of machine learning with other fields like quantum computing. It helps readers understand where the field is headed and how to stay up-to-date with the latest advancements.
Also filed under Math, Computer Science
Fundamentals of Machine Learning
Roozbeh Sanaei · Report
11 3.1.2 Different Algorithms in ICA . 14 3.1.5 Fast Independent Component Analysis . 17 3.2.2 Comparative Analysis of SNE, t-SNE, and UMAP . 18 3.2.3 SNE, t-SNE and UMAP Comparison .
Also filed under Math, Computer Science
Generative Adversarial Nets
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio · Report
The 2014 paper that introduced GANs — a framework where a generator and discriminator are trained in a minimax game. Opened the era of modern generative modeling and led directly to the image synthesis breakthroughs of the late 2010s.
Also filed under AI, Statistics
Gradient-Based Learning Applied to Document Recognition
LeCun et al. · Paper
Seminal paper by LeCun et al. (1998). Access: open-access. Source: http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton (University of Toronto) · Report
The 2012 AlexNet paper that catalyzed the deep learning revolution. A deep CNN trained on two GPUs reduced the ImageNet top-5 error rate from 26% to 15%, convincing the broader ML community that deep networks were production-ready.
Also filed under AI, Computer Science
ImageNet Classification with Deep Convolutional Neural Networks
Krizhevsky, Sutskever, and Hinton · Paper
Seminal paper by Krizhevsky, Sutskever, and Hinton (2012). Access: open-access. Source: https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
Introduction to Statistical and Machine Learning Methods for Data Science
Carlos Andre Reis Pinheiro & Mike Patetta · Book
"Introduction to Statistical and Machine Learning Methods for Data Science" by Carlos Andre Reis Pinheiro & Mike Patetta provides a comprehensive guide to the foundational techniques used in modern data science. The book bridges the gap between traditional statistical approaches and contemporary machine learning methods, offering readers a well-rounded understanding of data-driven decision-making. Key Topics Covered: Fundamentals of Statistical Methods: Covers core concepts such as probability distributions, hypothesis testing, regression analysis, and statistical inference. Machine Learning Techniques: Introduces supervised and unsupervised learning algorithms, including decision trees, random forests, support vector machines (SVMs), and neural networks. Feature Engineering & Data Preprocessing: Explores methods to clean, transform, and optimize data for machine learning models. Model Evaluation & Validation: Discusses key metrics like accuracy, precision, recall, and ROC curves to assess model performance. Real-World Applications: Demonstrates how statistical and machine learning methods are applied in finance, healthcare, and business analytics. This book is designed for data scientists, analysts, and professionals looking to enhance their knowledge of both classical statistical methods and modern machine learning techniques. With practical examples and code implementations, it serves as an essential resource for anyone aiming to leverage data science in their field.
Also filed under Statistics, Math
Language Models are Few-Shot Learners (GPT-3)
Tom B. Brown et al. (OpenAI) · Report
The 2020 GPT-3 paper demonstrating that scale alone — 175 billion parameters — unlocks few-shot in-context learning. The paper that transformed the commercial viability of large language models and kicked off the modern LLM era.
Also filed under AI
Large Language Models in Cybersecurity
Andrei Kucharavy, Octave Plancherel, Valentin Mulder, Alain Mermoud & Vincent Lenders · Book
This book is an open access publication. The images or other third party material in this book are included in the book’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. The use of general descriptive names, registered names, trademarks, service marks, etc.
Also filed under Risk Management, Computer Science
Learn Python Programming Quickly
Chris Ford · Book
Data Types in Python What Are Data Types? Understanding Data Types Type Casting 5. Lists and Tuples What Is a List Data Type?
Also filed under Python, Data Visualization
Learn SQL Fast
D Armstrong · Book
"Learn SQL Fast: SQL Made Simple! A Beginner's Guide to SQL, with Practical Exercises for Microsoft SQL Server" by D. Armstrong is a comprehensive resource designed to take readers from beginner to proficient in SQL. The book provides a step-by-step approach to learning SQL, focusing on practical application and hands-on exercises. Key Features: Guided Setup: The book begins by assisting readers in acquiring and setting up free SQL Server software and a sample database directly from Microsoft, ensuring a solid foundation for practice. Incremental Learning: SQL concepts are introduced in manageable segments, with each new topic accompanied by clear explanations and examples. This methodical approach facilitates a deeper understanding of each concept before progressing. Hands-On Exercises: At each stage, the book provides practice exercises, allowing readers to apply what they've learned and reinforce their skills through practical application. Progressive Complexity: As the course advances, examples and exercises become more complex, demonstrating how to combine basic concepts to write advanced queries. This progression ensures that readers build upon their knowledge systematically. This book is tailored for individuals who aim to learn SQL efficiently and effectively, providing both the practical skills and the underlying principles necessary for proficient database management.
Also filed under SQL, Data Visualization
Learn SQL Quickly
Code Quickly · Book
"Learn SQL Quickly: A Beginner’s Guide to Learning SQL, Even If You’re New to Databases" is a comprehensive introduction to SQL designed for readers without prior coding experience. Authored by Code Quickly, the book aims to demystify SQL and equip readers with practical skills for managing and manipulating databases. Key Features: Clear and Concise Explanations: The book offers straightforward examples to help readers grasp essential SQL concepts, ensuring a solid foundational understanding. Foundational Knowledge: It provides an in-depth breakdown of what SQL is, making complex topics accessible to beginners. Database Management Setup: Readers are guided through setting up their database management systems, emphasizing a methodical approach to ensure a strong grasp of the basics. Efficient Data Handling: The book teaches how to control data movement effectively, optimizing workflows and minimizing potential issues. Real-World Applications: Incorporating practical examples, the book demonstrates how SQL skills can be applied in real-life scenarios, enhancing learning and retention. Focus on MySQL: It discusses the advantages of using MySQL over other database systems and provides guidance on getting started with it. In today's data-driven world, proficiency in SQL is increasingly valuable. "Learn SQL Quickly" serves as a practical resource for those eager to acquire this in-demand skill efficiently.
Also filed under SQL, Data Visualization
Learning Algorithms and Market Manipulation
Álvaro Cartea · Slides
A learning algorithm consists of An objective, e.g., maximise profits, minimise costs. Offline with historical data Online as market evolves Both: offline and online What key aspects affect the performance of the learnt strategies?
Also filed under Quant Finance, Risk Management
Learning Internal Representations by Error Propagation
Rumelhart, Hinton, and Williams · Paper
Seminal paper by Rumelhart, Hinton, and Williams (1986). Access: open-access. Source: https://www.cs.toronto.edu/~hinton/absps/naturebp.pdf
Learning Node.js
Stack Overflow Documentation · Book
Learning Node.js (Stack Overflow Documentation) is a practical guide that compiles community-driven knowledge from Stack Overflow, offering a hands-on approach to mastering Node.js. This book is designed for developers of all levels who want to build scalable and high-performance applications using JavaScript on the server side. Key Highlights: Introduction to Node.js Explains the event-driven, non-blocking I/O model and why Node.js is ideal for real-time applications. Covers installation, setting up a development environment, and running a basic Node.js script. Core Node.js Modules and APIs In-depth coverage of built-in modules like fs (File System), http, events, path, os, and util. Working with buffers, streams, and process management. Asynchronous Programming Explains callbacks, Promises, and async/await to handle asynchronous operations effectively. Covers best practices for avoiding callback hell and improving code readability. Building Web Servers with Node.js Creating a basic HTTP server using the http module. Introduction to Express.js, a popular framework for building web applications and APIs. Working with Databases Connecting Node.js to relational (MySQL, PostgreSQL) and NoSQL (MongoDB) databases. Performing CRUD operations and handling database connections efficiently.
Also filed under Software Engineering, SQL
Learning Pandas
Stack Overflow Documentation · Book
Learning Pandas (Stack Overflow Documentation) is a practical and comprehensive guide to the Pandas library, offering a structured approach to data manipulation and analysis in Python. Designed for data analysts, scientists, and developers, this book compiles expert knowledge from Stack Overflow, providing clear explanations and hands-on examples to enhance data processing skills. Key Highlights: Foundational Understanding of Pandas : Introduces Pandas' core data structures, Series and DataFrames, demonstrating their fundamental role in data analysis. Data Importing and Exporting: Covers techniques for reading and writing data in various formats, including CSV, Excel, JSON, and SQL, ensuring seamless integration with different data sources. Data Cleaning and Transformation: Guides readers through handling missing values, filtering, indexing, and data type conversions, essential for preparing datasets for analysis. Efficient Data Manipulation: Explains powerful Pandas functions such as groupby(), merge(), pivot(), and apply(), enabling effective aggregation, reshaping, and computation on datasets. Exploratory Data Analysis (EDA): Discusses summary statistics, value distributions, and data visualization using Pandas, Matplotlib, and Seaborn, aiding in uncovering key insights. Advanced Techniques and Performance Optimization: Introduces time series analysis, vectorization, and efficient memory usage, helping users work with large datasets effectively. Real-World Applications: Provides practical examples and workflows for finance, machine learning preprocessing, and business analytics, making Pandas a valuable tool across industries. With a practical, example-driven approach, this book equips readers with the knowledge to harness the full power of Pandas for data analysis, making complex tasks more manageable and efficient. Whether you’re a beginner or an experienced user, this guide serves as an essential reference for mastering Pandas in Python.
Also filed under Python, Data Visualization
Lecture 2: Prediction
Alexandra Chouldechova · Slides
Lecture 2: Prediction Part I: Splines, Additive Models Part II: Model Selection and Validation Prof.
Also filed under Statistics, Risk Management
Lecture Notes On Artificial Intelligence
Prashanta Kumar Patra · Book
"Lecture Notes on Artificial Intelligence" is a structured and concise guide designed to introduce the fundamental concepts of Artificial Intelligence (AI). The book is tailored for students, researchers, and professionals looking to understand the theoretical foundations and practical applications of AI. It provides a systematic overview of AI methodologies, techniques, and algorithms, making it an essential resource for academic learning and real-world problem-solving. Key Topics Covered: Introduction to Artificial Intelligence: Definition, history, and evolution of AI, along with its impact on various industries. Problem Solving & Search Algorithms: Covers uninformed search (BFS, DFS), heuristic search (A), and optimization techniques*. Knowledge Representation & Reasoning: Explores logical reasoning, semantic networks, ontologies, and rule-based systems. Machine Learning Fundamentals: Introduces supervised, unsupervised, and reinforcement learning, along with key algorithms like decision trees and neural networks. Natural Language Processing (NLP): Discusses text processing, sentiment analysis, and AI-driven language models. Expert Systems & Fuzzy Logic: Explains the design and implementation of expert systems and how fuzzy logic enhances decision-making. Neural Networks & Deep Learning: Covers perceptrons, backpropagation, convolutional neural networks (CNNs), and recurrent neural networks (RNNs). AI Ethics & Future Trends: Discusses ethical considerations, AI governance, and the future impact of AI on society. With clear explanations, mathematical foundations, and practical examples, this book serves as a valuable resource for understanding AI principles and preparing for advanced studies or careers in artificial intelligence.
Also filed under AI, Computer Science
LLaMA: Open and Efficient Foundation Language Models
Hugo Touvron et al. (Meta AI) · Report
The 2023 paper releasing LLaMA — a family of 7B–65B parameter models trained on publicly available data. LLaMA-13B outperformed GPT-3 (175B) on most benchmarks. Opened the era of capable open-weight LLMs and enabled the explosion of open-source LLM work.
Also filed under AI
Machine Learning in Finance
Matthew F. Dixon, Igor Halperin, Paul Bilokon · Book
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for...
Also filed under Quant Finance, Finance
Machine Learning in Python For Everyone
Jonathan Wayna Korn · Book
An introduction to implementing machine learning algorithms in Python using libraries such as scikit-learn. It covers key concepts including data preprocessing, model training, evaluation metrics, and common supervised and unsupervised learning techniques.
Also filed under Python, Data Visualization
Machine Learning with Python
Tutorials Point · Book
A comprehensive tutorial from Tutorials Point covering the fundamentals of machine learning using Python. It introduces key ML concepts, the Python ecosystem for data science, methods and tasks suited for machine learning, data loading and preprocessing, and practical implementation of algorithms using NumPy, Scikit-learn, SciPy, and Matplotlib.
Also filed under Python, Computer Science
Mathematics for Inference and Machine Learning
Marc Deisenroth & Stefanos Zafeiriou · Report
5 1.2.1 Means and Covariances . 6 1.2.1.1 Sum of Random Variables . 7 1.2.1.2 Affine Transformation . 7 1.2.2 Statistical Independence .
Also filed under Statistics, Math
Mathematics For Machine Learning
Marc Peter Deisenroth, A. Aldo Faisal & Cheng Soon Ong · Book
"Mathematics for Machine Learning" is a comprehensive guide that provides the mathematical foundation necessary for understanding and developing machine learning models. The book is designed for students, engineers, and researchers who want to strengthen their mathematical skills to effectively engage with modern machine learning techniques. Key Topics Covered: Linear Algebra: Covers essential concepts such as vectors, matrices, eigenvalues, and singular value decomposition (SVD), which are crucial for algorithms like PCA and deep learning. Analytical Geometry: Explores coordinate transformations, basis changes, and geometric interpretations of machine learning problems. Matrix Decompositions: Discusses LU, QR, and Cholesky decompositions and their applications in optimization and numerical computations. Vector Calculus: Introduces gradients, Hessians, and Jacobians, which are fundamental for optimization and backpropagation in neural networks. Probability and Statistics: Provides an overview of probability distributions, Bayes’ theorem, and information theory concepts used in probabilistic machine learning models. Optimization Techniques: Explains gradient descent, convex optimization, and constrained optimization, which are essential for training machine learning models. The book balances theory with practical applications, making complex mathematical concepts more accessible through intuitive explanations and visualizations. By focusing on the mathematical principles behind machine learning, it equips readers with the tools needed to develop, analyze, and optimize machine learning algorithms effectively.
Also filed under Math, Quant Finance
Pattern Recognition and Machine Learning
Christopher M. Bishop · Book
Pattern Recognition and Machine Learning by Christopher M. Bishop is a fundamental textbook that provides a comprehensive introduction to statistical pattern recognition and machine learning. The book is well-regarded for its probabilistic approach, making it an essential resource for students, researchers, and practitioners in data science, artificial intelligence, and related fields. Key Highlights: Foundations of Pattern Recognition Introduces the Bayesian approach to pattern recognition, emphasizing probability distributions and decision theory. Covers fundamental concepts such as linear regression, classification, and generative models. Probability and Machine Learning Discusses core probability distributions, including Gaussian, Multivariate Gaussian, and Exponential families. Explains Bayesian inference, its role in machine learning, and its advantages over frequentist methods. Supervised Learning Techniques Covers linear and logistic regression, support vector machines (SVMs), and neural networks. Introduces regularization techniques to prevent overfitting. Unsupervised Learning and Clustering Explores principal component analysis (PCA), mixture models, and expectation-maximization (EM) algorithms. Discusses clustering techniques such as k-means and Gaussian mixture models (GMMs). Graphical Models and Probabilistic Inference Introduces Bayesian networks and Markov random fields for modeling dependencies between variables. Covers approximate inference techniques, including variational inference and Markov chain Monte Carlo (MCMC). Kernel Methods and Advanced Topics Discusses kernel-based learning, including Gaussian processes and support vector machines (SVMs). Covers hidden Markov models (HMMs) and their applications in sequential data analysis.
Also filed under Computer Science, Statistics
Python Automation Cookbook
Jaime Buelta · Book
"Python Automation Cookbook" by Jaime Buelta is a practical guide that demonstrates how to automate various tasks using Python, aiming to enhance efficiency and productivity. The book employs a problem-solution approach, providing recipes to automate repetitive tasks across different domains. Key Features: Web Scraping and Data Extraction: Learn techniques to scrape websites, detect changes, and extract valuable information for analysis. File and Data Management: Discover methods to search, process, and aggregate raw data files into structured formats like spreadsheets. Report Generation: Explore ways to extract data from Excel spreadsheets and generate comprehensive reports with graphs using libraries such as Matplotlib. Marketing Automation: Understand how to automatically generate marketing campaigns and communicate with recipients over different channels. Debugging Techniques: Gain insights into identifying and implementing precise solutions to common automation challenges. The second edition of the book includes additional chapters focusing on automated code testing, machine learning projects, and handling unstructured data, reflecting Python's growth in data science and AI automation. This cookbook is suitable for developers, data enthusiasts, or anyone interested in automating monotonous manual tasks related to business processes such as finance, sales, and HR. It serves both as a step-by-step guide for beginners and a reference for experienced Python users seeking to streamline their workflows.
Also filed under Python, Data Visualization
Python By Example
Nichola Lacey · Book
"Python By Example" by Nichola Lacey provides a practical approach to learning Python programming through hands-on examples. It guides readers from basic concepts to more advanced topics, using real-world scenarios to illustrate how Python can be applied effectively. The book is designed to help beginners understand and implement Python code by working through a series of projects and exercises.
Also filed under Python, Data Visualization
Python Fast/Deep/Simple
Behnam Khani · Book
About the author Hello My name is Behnam Khani. I’m a software engineer with 10 years of experience in the industry. I have a passion for technology, education, and software development and enjoy combining the three. This book and my website dejavucode.com are places to share my knowledge and experiences!
Also filed under Python, Data Visualization
Python In easy steps
Mike McGrath · Book
No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without prior written permission from the publisher. Notice of Liability Every effort has been made to ensure that this book contains accurate and current information. However, In Easy Steps Limited and the author shall not be liable for any loss or damage suffered by readers as a result of any information contained herein. Trademarks All trademarks are acknowledged as belonging to their respective ...
Also filed under Python, Data Visualization
Python Machine Learning Workbook for Begginers
AI Publishing · Guide
Introduction and Environment Set Up Data science libraries exist in various programming languages. However, you will be using Python programming language for data science and machine learning since Python is flexible, easy to learn, and offers the most advanced data science and machine learning libraries. Furthermore, Python has a huge data science community from where you can take help whenever you want. In this chapter, you will see how to set up the Python environment needed to run various data science and machine learning libraries.
Also filed under Python, Data Visualization
Python Machine Learning: A Beginner's Guide to Scikit-Learn
Rajender Kumar · Book
No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the copyright owner. This book is sold subject to the condition that it shall not, by way of trade or otherwise, be lent, resold, hired out, or otherwise circulated without the publisher's prior consent in any form of binding or cover other than that in which it is published and without a similar condition including this condition being imposed on the subsequent purchaser. Trad...
Also filed under Python, Statistics
Python Pandas Tutorial For Beginners : The Ultimate Guide For Beginners
Kavi & Shila · Book
A beginner-friendly tutorial on the Pandas library in Python for data manipulation and analysis. It covers fundamental operations such as creating DataFrames, reading CSV files, filtering data, and performing basic statistical summaries.
Also filed under Python, Data Visualization
Python Programming Essentials
Ankit Pandey · Book
Emphasis will also be placed on verifying the installation and resolving potential issues that may arise during the process. Windows Installation To install Python on a Windows operating system, the official Python distribution is recommended. Select the Downloads section and choose the appropria
Also filed under Python, Data Visualization
Representation Learning for Natural Language Processing
Zhiyuan Liu, Yankai Lin & Maosong Sun · Book
Preface In conventional natural language processing (NLP) systems, language items such as words and phrases are handled as distinct symbols. Many classical methods, such as n-gram and bag-of-words models, were proposed and have been widely used until now. All these methods take words as the minimum units for semantic representation, either used to estimate the conditional probabilities of the next word given previous words (e.g., n-gram) or used to represent semantic meanings of text (e.g., bag-of-words models). Even when people find it necessary to model word meanings, they either manually ...
Also filed under AI, Data Visualization
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Patrick Lewis et al. (Facebook AI Research) · Report
The 2020 paper that formalized RAG — combining a parametric seq2seq model with a non-parametric retrieval component. The architecture underlying most production LLM deployments that need current or private knowledge.
Also filed under AI
Rossmann Sales Prediction Using Supervised Models
Nikita Prasad · Report
No encontrado
Also filed under Risk Management, Quant Finance
Scaling Laws for Neural Language Models
Jared Kaplan et al. (OpenAI) · Report
The 2020 Kaplan paper establishing empirical scaling laws for language models: loss is predictable as a power law in model size, dataset size, and compute. The paper that convinced OpenAI to bet on scale and led directly to GPT-3.
Also filed under AI, Statistics
The Hundred-Page Machine Learning Book
Andriy Burkov · Book
"The Hundred-Page Machine Learning Book" by Andriy Burkov provides a concise overview of machine learning concepts, techniques, and algorithms. It covers fundamental topics such as supervised and unsupervised learning, model evaluation, and neural networks, offering practical insights and examples to help readers understand and apply machine learning in real-world scenarios.
Also filed under Math, Computer Science
The legal implications of Generative AI
Donna Bartlett, Willem-Jan Cosemans, Matt Saunders, Till Contzen, Klaus Gresbrand, Maria-Alexandra Papoutsi, Pietro Boccaccini, Peggy Anstett, Bruce Braude, Richard Reeve-Young · Report
The book explores the legal challenges and considerations surrounding the use of generative AI technologies. It examines issues such as intellectual property rights, data privacy, liability, and regulatory frameworks. The authors provide insights into how these technologies impact various industries and offer guidance on navigating the evolving legal landscape to ensure compliance and mitigate risks.
The Lottery Ticket Hypothesis
Frankle and Carbin · Paper
Seminal paper by Frankle and Carbin (2018). Access: open-access. Source: https://arxiv.org/abs/1803.03635
Toolformer: Language Models Can Teach Themselves to Use Tools
Timo Schick et al. (Meta AI) · Report
The 2023 Toolformer paper showing how LLMs can be self-taught to invoke external APIs (calculator, search, translation) by inserting API call annotations into training data. A foundational paper in the agentic-LLM lineage.
Also filed under AI
Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts
Maciej Besta, Florim Memedi, Zhenyu Zhang, Robert Gerstenberger, Nils Blach, Piotr Nyczyk, Marcin Copik, Grzegorz Kwasniewski, Jurgen Muller, Lukas Gianinazzi, Ales Kubicek, Hubert Niewiadomski, Onur Mutlu,Torsten Hoefler · Report
"Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts" is a comprehensive study by Maciej Besta, Florim Memedi, Zhenyu Zhang, Robert Gerstenberger, Nils Blach, Piotr Nyczyk, Marcin Copik, Grzegorz Kwaśniewski, Jürgen Müller, Lukas Gianinazzi, Ales Kubicek, Hubert Niewiadomski, Onur Mutlu, and Torsten Hoefler, published in January 2024. The paper explores structured prompting techniques in large language models (LLMs), focusing on how different reasoning topologies—chains, trees, and graphs—can enhance model performance across various tasks. Key Highlights: Structured Prompting Techniques: The study introduces paradigms like Chain-of-Thought, Tree of Thoughts, and Graph of Thoughts, which guide LLM reasoning through explicit structures, leading to improved task-solving capabilities. Taxonomy of Reasoning Topologies: The authors present a taxonomy categorizing structure-enhanced LLM reasoning schemes, analyzing aspects such as topology class, scope, representation, and execution algorithms. Comparative Analysis: By dissecting existing prompting schemes, the paper evaluates how design choices influence performance metrics like accuracy, latency, and cost, providing insights into effective prompting strategies. Theoretical Insights and Future Challenges: The paper discusses the interplay between prompting techniques and other LLM components, highlighting research challenges and suggesting directions for future advancements in prompt engineering. This work serves as a valuable resource for understanding and advancing structured prompting methods in LLMs, offering a foundation for developing more efficient and effective language models.
Training Compute-Optimal Large Language Models (Chinchilla)
Jordan Hoffmann et al. (DeepMind) · Report
The 2022 Chinchilla paper that corrected Kaplan's scaling laws. Trained 400+ models from 70M to 16B parameters, showing that model size and training tokens should scale equally — a 70B Chinchilla with 4× more data beat 280B Gopher. Reshaped the training strategy for every subsequent frontier LLM.
Also filed under AI, Statistics
Training Language Models to Follow Instructions with Human Feedback (InstructGPT)
Long Ouyang et al. (OpenAI) · Report
The 2022 InstructGPT paper establishing the RLHF recipe: supervised fine-tuning on high-quality demonstrations, reward model training from human preferences, and PPO optimization. The 1.3B InstructGPT beat the 175B GPT-3 on human preference — proof that alignment mattered more than scale.
Also filed under AI
U-Net
Ronneberger, Fischer, and Brox · Paper
Seminal paper by Ronneberger, Fischer, and Brox (2015). Access: open-access. Source: https://arxiv.org/abs/1505.04597
Unlocking Artificial Intelligence
Christopher Mutschler, Christian Münzenmayer, Norman Uhlmann & Alexander Martin · Book
This book is an open access publication. The images or other third party material in this book are included in the book’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. The use of general descriptive names, registered names, trademarks, service marks, etc.
The documents are the work of their respective authors and publishers; Gyre Research claims no ownership and will remove any document on request from a rightsholder — team@gyreresearch.com. The summaries and subject classifications are original work by Gyre Research and may be quoted with attribution.