NoSQL, Big Data, and Spark Foundations

Differentiate between the four main categories of NoSQL repositories. Describe the characteristics, features, benefits, limitations, and applications of the more popular Big Data processing tools. Perform common tasks using MongoDB tasks including create, read, update, and delete (CRUD) operations. Execute keyspace, table, and CRUD operations in Cassandra.

Explain the impact of big data, including use cases, tools, and processing methods. Describe Apache Hadoop architecture, ecosystem, practices, and user-related applications, including Hive, HDFS, HBase, Spark, and MapReduce. Apply Spark programming basics, including parallel programming basics for DataFrames, data sets, and Spark SQL. Use Spark’s RDDs and data sets, optimize Spark SQL using Catalyst and Tungsten, and use Spark’s development and runtime environment options.

Describe ML, explain its role in data engineering, summarize generative AI, discuss Spark's uses, and analyze ML pipelines and model persistence. Evaluate ML models, distinguish between regression, classification, and clustering models, and compare data engineering pipelines with ML pipelines. Construct the data analysis processes using Spark SQL, and perform regression, classification, and clustering using SparkML. Demonstrate connecting to Spark clusters, build ML pipelines, perform feature extraction and transformation, and model persistence.