📌 Mongoose Aggregation & Aggregation Pipeline

Well-Structured Notes with Examples

📖 Introduction to Mongoose Aggregation

Mongoose provides an aggregation framework to perform advanced data processing on MongoDB collections. Aggregation allows grouping, filtering, sorting, transforming, and analyzing data efficiently.

🏗 Key Features of Aggregation

Processes large amounts of data in a single query.
Supports grouping, filtering, sorting, and transforming data.
Reduces the need for multiple queries.
Improves performance by reducing network overhead.

🛠 Aggregation Pipeline in Mongoose

The Aggregation Pipeline is a sequence of stages that process documents. Each stage takes input, processes it, and passes the output to the next stage.

📌 Common Aggregation Stages

Stage	Description
`$match`	Filters documents based on conditions (like `find()`).
`$group`	Groups documents by a specified field and performs operations like sum, avg, min, max, etc.
`$project`	Specifies which fields to include or exclude.
`$sort`	Sorts documents based on field values.
`$limit`	Limits the number of documents returned.
`$skip`	Skips a specified number of documents.
`$lookup`	Performs joins between collections.
`$unwind`	Deconstructs an array field into separate documents.
`$addFields`	Adds new computed fields.
`$count`	Counts the number of documents.
`$facet`	Runs multiple aggregation pipelines in parallel.

📌 Example Data (`User` Collection)

Consider a MongoDB collection named "users":

[
  { "_id": 1, "name": "Alice", "age": 25, "city": "New York", "salary": 5000 },
  { "_id": 2, "name": "Bob", "age": 30, "city": "London", "salary": 7000 },
  { "_id": 3, "name": "Charlie", "age": 35, "city": "New York", "salary": 6500 },
  { "_id": 4, "name": "David", "age": 28, "city": "London", "salary": 7200 }
]

🎯 Aggregation Pipeline Examples

1️⃣ Using `$match` - Filter users from "New York"

const usersFromNY = await User.aggregate([
  { $match: { city: "New York" } }
]);
console.log(usersFromNY);

Output: Users from New York.

2️⃣ Using `$group` - Get Average Salary by City

const avgSalaryByCity = await User.aggregate([
  { $group: { _id: "$city", avgSalary: { $avg: "$salary" } } }
]);
console.log(avgSalaryByCity);

Output:

[
  { "_id": "New York", "avgSalary": 5750 },
  { "_id": "London", "avgSalary": 7100 }
]

3️⃣ Using `$sort` - Sort Users by Salary in Descending Order

const sortedUsers = await User.aggregate([
  { $sort: { salary: -1 } }
]);
console.log(sortedUsers);

Output: Users sorted by highest salary first.

4️⃣ Using `$project` - Show Only `name` and `salary`

const projectedUsers = await User.aggregate([
  { $project: { name: 1, salary: 1, _id: 0 } }
]);
console.log(projectedUsers);

Output:

[
  { "name": "Alice", "salary": 5000 },
  { "name": "Bob", "salary": 7000 },
  { "name": "Charlie", "salary": 6500 },
  { "name": "David", "salary": 7200 }
]

5️⃣ Using `$lookup` - Join Users with Orders Collection

const usersWithOrders = await User.aggregate([
  {
    $lookup: {
      from: "orders", // Collection to join
      localField: "_id", // Field in 'users'
      foreignField: "userId", // Field in 'orders'
      as: "orders" // Output array field
    }
  }
]);
console.log(usersWithOrders);

Output: Each user document will have an orders array.

6️⃣ Using `$unwind` - Flatten Array Fields

const unwoundUsers = await User.aggregate([
  { $unwind: "$orders" }
]);
console.log(unwoundUsers);

Output: Each user will have multiple documents, each with a single order.

7️⃣ Using `$count` - Count Total Users

const totalUsers = await User.aggregate([
  { $count: "totalUsers" }
]);
console.log(totalUsers);

Output:

[
  { "totalUsers": 4 }
]

8️⃣ Using `$facet` - Multiple Aggregations in One Query

const userStats = await User.aggregate([
  {
    $facet: {
      ageStats: [
        { $group: { _id: null, avgAge: { $avg: "$age" } } }
      ],
      salaryStats: [
        { $group: { _id: null, maxSalary: { $max: "$salary" }, minSalary: { $min: "$salary" } } }
      ]
    }
  }
]);
console.log(userStats);

Output:

{
  "ageStats": [{ "avgAge": 29.5 }],
  "salaryStats": [{ "maxSalary": 7200, "minSalary": 5000 }]
}

🏆 Key Takeaways

✅ Aggregation allows efficient data processing with multiple transformations.
✅ The Aggregation Pipeline uses stages like $match, $group, $lookup, etc.
✅ Mongoose aggregate() method provides easy access to MongoDB’s powerful aggregation features.
✅ Joins, filtering, sorting, and computation can all be done in a single query, reducing server load.

🚀 Final Thoughts

Mongoose Aggregation is powerful for analytics and reporting. Use it when you need complex data manipulations beyond simple find() queries.

🔥 Advanced Aggregation Examples

9️⃣ Using `$addFields` - Add a Computed Field

Add a new field annualSalary which is 12 times the salary:

const usersWithAnnualSalary = await User.aggregate([
  {
    $addFields: {
      annualSalary: { $multiply: ["$salary", 12] }
    }
  }
]);
console.log(usersWithAnnualSalary);

Output:

[
  { "name": "Alice", "salary": 5000, "annualSalary": 60000 },
  { "name": "Bob", "salary": 7000, "annualSalary": 84000 },
  { "name": "Charlie", "salary": 6500, "annualSalary": 78000 },
  { "name": "David", "salary": 7200, "annualSalary": 86400 }
]

🔟 Using `$redact` - Conditionally Hide Data

Hide users with a salary less than 6000:

const filteredUsers = await User.aggregate([
  {
    $redact: {
      $cond: {
        if: { $gte: ["$salary", 6000] },
        then: "$$KEEP",
        else: "$$PRUNE"
      }
    }
  }
]);
console.log(filteredUsers);

Output: Only users with salary ≥ 6000 will be shown.

1️⃣1️⃣ Using `$bucket` - Group Users into Age Ranges

Group users into different age brackets:

const ageGroups = await User.aggregate([
  {
    $bucket: {
      groupBy: "$age",
      boundaries: [20, 30, 40], // Age ranges (20-29, 30-39)
      default: "40+", // Group ages 40+ into one
      output: {
        count: { $sum: 1 },
        avgSalary: { $avg: "$salary" }
      }
    }
  }
]);
console.log(ageGroups);

Output: Groups users into 20-29, 30-39, and 40+ categories.

1️⃣2️⃣ Using `$out` - Save Aggregation Result in a New Collection

Store the aggregated result into a new MongoDB collection:

await User.aggregate([
  {
    $group: { _id: "$city", totalUsers: { $sum: 1 } }
  },
  { $out: "user_summary" } // Creates a new collection named 'user_summary'
]);

Output: A new user_summary collection is created in the database.

1️⃣3️⃣ Using `$merge` - Merge Aggregated Data into an Existing Collection

Merge data into an existing collection:

await User.aggregate([
  {
    $group: { _id: "$city", avgSalary: { $avg: "$salary" } }
  },
  { $merge: { into: "city_salary_summary", whenMatched: "merge" } }
]);

Output: Merges results into city_salary_summary collection.

🎯 When to Use Aggregation?

Data transformation (e.g., adding new computed fields).
Filtering and grouping (e.g., by age, city, etc.).
Joining collections (e.g., users and orders).
Generating reports (e.g., average salary per city).
Optimizing performance by reducing multiple queries.

🔥 Advanced Mongoose Aggregation Examples

1️⃣4️⃣ Using `$text` - Full-Text Search

Perform full-text search using a text index:

// Step 1: Create a Text Index
await User.collection.createIndex({ name: "text", city: "text" });

// Step 2: Search for Users Matching a Keyword
const searchResults = await User.aggregate([
  { $match: { $text: { $search: "New York" } } }
]);
console.log(searchResults);

Output: Finds users whose name or city contains "New York".

1️⃣5️⃣ Using `$sample` - Get Random Users

Fetch random users:

const randomUsers = await User.aggregate([
  { $sample: { size: 2 } } // Get 2 random users
]);
console.log(randomUsers);

Output: Returns 2 randomly selected users.

1️⃣6️⃣ Using `$expr` - Compare Two Fields

Find users earning less than expected:

const underpaidUsers = await User.aggregate([
  { $match: { $expr: { $lt: ["$salary", "$expectedSalary"] } } }
]);
console.log(underpaidUsers);

Output: Lists users where salary < expectedSalary.

1️⃣7️⃣ Using `$map` - Modify an Array Field

Capitalize all skill names in the skills array:

const usersWithCapitalizedSkills = await User.aggregate([
  {
    $project: {
      name: 1,
      skills: {
        $map: {
          input: "$skills",
          as: "skill",
          in: { $toUpper: "$$skill" }
        }
      }
    }
  }
]);
console.log(usersWithCapitalizedSkills);

Output:

[
  { "name": "Alice", "skills": ["NODEJS", "MONGODB", "EXPRESS"] }
]

1️⃣8️⃣ Using `$filter` - Filter an Array Field

Keep only orders greater than $500:

const usersWithFilteredOrders = await User.aggregate([
  {
    $project: {
      name: 1,
      orders: {
        $filter: {
          input: "$orders",
          as: "order",
          cond: { $gt: ["$$order.amount", 500] }
        }
      }
    }
  }
]);
console.log(usersWithFilteredOrders);

Output:

[
  {
    "name": "Alice",
    "orders": [
      { "orderId": 102, "amount": 600 },
      { "orderId": 103, "amount": 900 }
    ]
  }
]

1️⃣9️⃣ Using `$setWindowFields` - Rank Users by Salary

Assign a rank to users based on salary:

const rankedUsers = await User.aggregate([
  {
    $setWindowFields: {
      sortBy: { salary: -1 },
      output: {
        rank: { $rank: {} }
      }
    }
  }
]);
console.log(rankedUsers);

Output: Assigns a rank to users based on salary.

2️⃣0️⃣ Using `$dateFromString` - Convert String to Date

Convert dob from string to Date type:

const usersWithDate = await User.aggregate([
  {
    $project: {
      name: 1,
      dob: { $dateFromString: { dateString: "$dob" } }
    }
  }
]);
console.log(usersWithDate);

Output: Converts dob from string to Date.

2️⃣1️⃣ Using `$geoNear` - Find Users Near a Location

Find users near New York:

const nearbyUsers = await User.aggregate([
  {
    $geoNear: {
      near: { type: "Point", coordinates: [-73.935242, 40.730610] }, // New York
      distanceField: "distanceFromNY",
      spherical: true
    }
  }
]);
console.log(nearbyUsers);

Output: Lists users sorted by distance from New York.

🎯 Key Takeaways

✅ $text → Full-text search.
✅ $sample → Get random documents.
✅ $expr → Compare two fields.
✅ $map → Modify array elements.
✅ $filter → Filter array elements.
✅ $setWindowFields → Ranking & running totals.
✅ $dateFromString → Convert string to date.
✅ $geoNear → Find nearest locations.

🚀 Final Thoughts

These advanced aggregation techniques allow data transformation, ranking, text search, and geospatial queries. Use them to unlock the full potential of MongoDB's aggregation framework!

📌 Mongoose Aggregation & Aggregation Pipeline (Well-Structured Notes with Examples)

📌 Mongoose Aggregation & Aggregation Pipeline

📖 Introduction to Mongoose Aggregation

🏗 Key Features of Aggregation

🛠 Aggregation Pipeline in Mongoose

📌 Common Aggregation Stages

📌 Example Data (User Collection)

🎯 Aggregation Pipeline Examples

1️⃣ Using $match - Filter users from "New York"

2️⃣ Using $group - Get Average Salary by City

3️⃣ Using $sort - Sort Users by Salary in Descending Order

4️⃣ Using $project - Show Only name and salary

5️⃣ Using $lookup - Join Users with Orders Collection

6️⃣ Using $unwind - Flatten Array Fields

7️⃣ Using $count - Count Total Users

8️⃣ Using $facet - Multiple Aggregations in One Query

🏆 Key Takeaways

🚀 Final Thoughts

🔥 Advanced Aggregation Examples

9️⃣ Using $addFields - Add a Computed Field

🔟 Using $redact - Conditionally Hide Data

1️⃣1️⃣ Using $bucket - Group Users into Age Ranges

1️⃣2️⃣ Using $out - Save Aggregation Result in a New Collection

1️⃣3️⃣ Using $merge - Merge Aggregated Data into an Existing Collection

🎯 When to Use Aggregation?

🔥 Advanced Mongoose Aggregation Examples

1️⃣4️⃣ Using $text - Full-Text Search

1️⃣5️⃣ Using $sample - Get Random Users

1️⃣6️⃣ Using $expr - Compare Two Fields

1️⃣7️⃣ Using $map - Modify an Array Field

1️⃣8️⃣ Using $filter - Filter an Array Field

1️⃣9️⃣ Using $setWindowFields - Rank Users by Salary

2️⃣0️⃣ Using $dateFromString - Convert String to Date

2️⃣1️⃣ Using $geoNear - Find Users Near a Location

🎯 Key Takeaways

🚀 Final Thoughts

📌 Example Data (`User` Collection)

1️⃣ Using `$match` - Filter users from "New York"

2️⃣ Using `$group` - Get Average Salary by City

3️⃣ Using `$sort` - Sort Users by Salary in Descending Order

4️⃣ Using `$project` - Show Only `name` and `salary`

5️⃣ Using `$lookup` - Join Users with Orders Collection

6️⃣ Using `$unwind` - Flatten Array Fields

7️⃣ Using `$count` - Count Total Users

8️⃣ Using `$facet` - Multiple Aggregations in One Query

9️⃣ Using `$addFields` - Add a Computed Field

🔟 Using `$redact` - Conditionally Hide Data

1️⃣1️⃣ Using `$bucket` - Group Users into Age Ranges

1️⃣2️⃣ Using `$out` - Save Aggregation Result in a New Collection

1️⃣3️⃣ Using `$merge` - Merge Aggregated Data into an Existing Collection

1️⃣4️⃣ Using `$text` - Full-Text Search

1️⃣5️⃣ Using `$sample` - Get Random Users

1️⃣6️⃣ Using `$expr` - Compare Two Fields

1️⃣7️⃣ Using `$map` - Modify an Array Field

1️⃣8️⃣ Using `$filter` - Filter an Array Field

1️⃣9️⃣ Using `$setWindowFields` - Rank Users by Salary

2️⃣0️⃣ Using `$dateFromString` - Convert String to Date

2️⃣1️⃣ Using `$geoNear` - Find Users Near a Location