================================================================================
  Question CQ3: MQTT Telemetry & Decision Tree Room Occupancy Classifier [30 Marks]
================================================================================

FULL QUESTION:
CAE 2 IoT Practical Exam - Combined Question:
CQ3. MQTT-Based Room Telemetry Streaming and Decision Tree Occupancy Classification [30 Marks]
Develop an edge-to-cloud IoT smart office monitoring pipeline using the MQTT protocol and a Decision Tree Classifier.
Input Parameters: Temperature, Relative Humidity, Light (Lux), and CO2 (ppm).
Historical Dataset: UCI Room Occupancy Detection dataset (datatraining.txt).
Target Variable: Occupancy (0 = Not Occupied, 1 = Occupied).
The system should:
- Implement an MQTT Publisher that streams simulated/edge room sensor telemetry in JSON format to a public broker (HiveMQ or Mosquitto). [6 Marks]
- Implement an MQTT Subscriber using paho-mqtt that receives and parses incoming telemetry packets in real time. [6 Marks]
- Train a Decision Tree Classifier on the historical datatraining.txt dataset and evaluate its accuracy score and confusion matrix. [5 Marks]
- Perform real-time inference on every incoming MQTT payload and determine room occupancy status. [5 Marks]
- Develop a Streamlit web dashboard displaying live MQTT telemetry feeds, dynamic occupancy status (Occupied / Vacant), and live charts. [5 Marks]
- Provide visual alerts and model evaluation metrics on the dashboard. [3 Marks]
Flow: Sensor Publisher -> MQTT Broker (HiveMQ:1883) -> MQTT Subscriber / Classifier -> Streamlit Dashboard.

SHORT QUESTION:
MQTT Room Telemetry Publisher & Subscriber with Decision Tree Occupancy Classifier and Streamlit Dashboard [30 Marks].

================================================================================
README & SETUP INSTRUCTIONS:
================================================================================
# Question CQ3: MQTT-Based Room Occupancy Classifier

## Problem Statement [30 Marks]
Develop an MQTT publish/subscribe telemetry stream and apply a Decision Tree Classifier to predict room occupancy based on Temperature, Humidity, Light, and CO2.

- **Broker**: `broker.hivemq.com` (Port 1883)
- **Topic**: `iot_lab/room/telemetry`
- **Dataset**: `datatraining.txt` (or synthetic fallback)
- **Target**: `Occupancy` (0 or 1)
- **Model**: Decision Tree Classifier (`scikit-learn`)

---

## Architectural Flow
```text
Sensor Publisher ──> MQTT Broker (HiveMQ:1883) ──> MQTT Subscriber ──> Decision Tree ──> Streamlit UI
```

---

## Files in this Package
- `publisher.py`: Simulates periodic multi-sensor room telemetry and publishes JSON payloads to the MQTT broker.
- `subscriber.py`: CLI Paho-MQTT subscriber that trains the Decision Tree and outputs live predictions as packets arrive.
- `app.py`: Streamlit interactive dashboard with live MQTT listener, real-time gauges, and occupancy badge.

---

## How to Run

### Step 1: Install Dependencies
```bash
pip install pandas scikit-learn streamlit paho-mqtt matplotlib
```

### Step 2: Start the MQTT Subscriber OR Streamlit App
Open Terminal 1:
```bash
python subscriber.py
```

### Step 3: Start the MQTT Publisher
Open Terminal 2:
```bash
python publisher.py
```

### Step 4: Run Streamlit Web Dashboard (Alternative Option)
Open Terminal 3:
```bash
streamlit run app.py
```

================================================================================
SOURCE CODE FILES:
================================================================================

1. publisher.py
--------------------------------------------------------------------------------
import time
import json
import random
import paho.mqtt.client as mqtt

BROKER = "broker.hivemq.com"
PORT = 1883
TOPIC = "iot_lab/room/telemetry"

def create_client():
    try:
        client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
    except AttributeError:
        client = mqtt.Client()
    return client

def main():
    client = create_client()
    print("=" * 60)
    print("  CQ3: MQTT Room Telemetry Publisher")
    print("=" * 60)
    print(f"Connecting to Broker: {BROKER}:{PORT}")

    try:
        client.connect(BROKER, PORT, keepalive=60)
        client.loop_start()
        print(f"Publishing room telemetry to topic: {TOPIC}\n")

        cycle = 0
        while True:
            cycle += 1

            is_occupied = (cycle // 5) % 2 == 1

            if is_occupied:
                payload = {
                    "Temperature": round(random.uniform(21.8, 24.2), 2),
                    "Humidity": round(random.uniform(26.0, 34.0), 2),
                    "Light": round(random.uniform(380.0, 620.0), 1),
                    "CO2": round(random.uniform(700.0, 1350.0), 1),
                    "timestamp": time.strftime("%Y-%m-%d %H:%M:%S")
                }
            else:
                payload = {
                    "Temperature": round(random.uniform(19.2, 21.0), 2),
                    "Humidity": round(random.uniform(20.0, 27.0), 2),
                    "Light": round(random.uniform(0.0, 45.0), 1),
                    "CO2": round(random.uniform(410.0, 510.0), 1),
                    "timestamp": time.strftime("%Y-%m-%d %H:%M:%S")
                }

            msg = json.dumps(payload)
            client.publish(TOPIC, msg)
            print(f"[MQTT Pub] Sent Payload #{cycle} -> {msg}")
            time.sleep(3)

    except KeyboardInterrupt:
        print("\n[Publisher] Stopping publisher...")
        client.loop_stop()
        client.disconnect()

if __name__ == "__main__":
    main()
--------------------------------------------------------------------------------

2. subscriber.py
--------------------------------------------------------------------------------
import os
import json
import time
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score, confusion_matrix
import paho.mqtt.client as mqtt

BROKER = "broker.hivemq.com"
PORT = 1883
TOPIC = "iot_lab/room/telemetry"
FEATURES = ["Temperature", "Humidity", "Light", "CO2"]
TARGET = "Occupancy"

def load_or_create_dataset():
    possible_paths = [
        "datatraining.txt",
        os.path.join(os.path.dirname(__file__), "datatraining.txt"),
        os.path.join("..", "datatraining.txt")
    ]
    for p in possible_paths:
        if os.path.exists(p):
            print(f"[Dataset] Loading UCI Occupancy dataset from {p}...")
            df = pd.read_csv(p)
            df.columns = [c.replace('"', '').strip() for c in df.columns]
            if all(col in df.columns for col in FEATURES + [TARGET]):
                return df[FEATURES + [TARGET]].dropna()

    print("[Dataset] Local datatraining.txt not found. Building synthetic dataset...")
    np.random.seed(42)
    n = 2000
    y = np.random.choice([0, 1], size=n)
    temp = np.where(y == 1, np.random.uniform(21.5, 24.5, n), np.random.uniform(19.0, 21.0, n))
    hum = np.where(y == 1, np.random.uniform(25.0, 35.0, n), np.random.uniform(20.0, 28.0, n))
    light = np.where(y == 1, np.random.uniform(300.0, 600.0, n), np.random.uniform(0.0, 50.0, n))
    co2 = np.where(y == 1, np.random.uniform(600.0, 1400.0, n), np.random.uniform(400.0, 520.0, n))
    return pd.DataFrame({"Temperature": temp, "Humidity": hum, "Light": light, "CO2": co2, "Occupancy": y})

print("Training Decision Tree model on historical data...")
df = load_or_create_dataset()
X_train, X_test, y_train, y_test = train_test_split(df[FEATURES], df[TARGET], test_size=0.20, random_state=42, stratify=df[TARGET])
model = DecisionTreeClassifier(max_depth=5, random_state=42)
model.fit(X_train, y_train)

acc = accuracy_score(y_test, model.predict(X_test))
cm = confusion_matrix(y_test, model.predict(X_test))
print(f"[Model] Accuracy: {acc*100:.2f}%")
print(f"[Model] Confusion Matrix:\n{cm}")

def on_message(client, userdata, message):
    try:
        payload_str = message.payload.decode("utf-8")
        data = json.loads(payload_str)
        t, h, l, c = data["Temperature"], data["Humidity"], data["Light"], data["CO2"]

        pred = model.predict([[t, h, l, c]])[0]
        prob = model.predict_proba([[t, h, l, c]])[0]

        status = "🔴 OCCUPIED" if pred == 1 else "🟢 VACANT"
        print(f"\n[MQTT Recv {data.get('timestamp','')}] T={t}°C | H={h}% | Light={l}lx | CO2={c}ppm")
        print(f"  ──> PREDICTED OCCUPANCY: {status} (Confidence: {max(prob)*100:.1f}%)")

    except Exception as e:
        print(f"[Subscriber Error] {e}")

def main():
    print("=" * 60)
    print("  CQ3: MQTT Subscriber & Decision Tree Classifier")
    print("=" * 60)

    try:
        client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2)
    except AttributeError:
        client = mqtt.Client()

    client.on_message = on_message
    print(f"Connecting to {BROKER}:{PORT} and subscribing to {TOPIC}...")
    client.connect(BROKER, PORT, keepalive=60)
    client.subscribe(TOPIC)

    try:
        client.loop_forever()
    except KeyboardInterrupt:
        print("\n[Subscriber] Disconnecting...")
        client.disconnect()

if __name__ == "__main__":
    main()
--------------------------------------------------------------------------------

3. app.py
--------------------------------------------------------------------------------
import os
import json
import time
import numpy as np
import pandas as pd
import streamlit as st
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score, confusion_matrix
import paho.mqtt.client as mqtt

st.set_page_config(page_title="CQ3: MQTT Occupancy Dashboard", layout="wide", page_icon="📡")

BROKER = "broker.hivemq.com"
PORT = 1883
TOPIC = "iot_lab/room/telemetry"
FEATURES = ["Temperature", "Humidity", "Light", "CO2"]
TARGET = "Occupancy"

@st.cache_resource
def get_trained_model():
    possible_paths = [
        "datatraining.txt",
        os.path.join(os.path.dirname(__file__), "datatraining.txt"),
        os.path.join("..", "datatraining.txt")
    ]
    df = None
    for p in possible_paths:
        if os.path.exists(p):
            df = pd.read_csv(p)
            df.columns = [c.replace('"', '').strip() for c in df.columns]
            if all(col in df.columns for col in FEATURES + [TARGET]):
                df = df[FEATURES + [TARGET]].dropna()
                break

    if df is None:
        np.random.seed(42)
        n = 2000
        y = np.random.choice([0, 1], size=n)
        temp = np.where(y == 1, np.random.uniform(21.5, 24.5, n), np.random.uniform(19.0, 21.0, n))
        hum = np.where(y == 1, np.random.uniform(25.0, 35.0, n), np.random.uniform(20.0, 28.0, n))
        light = np.where(y == 1, np.random.uniform(300.0, 600.0, n), np.random.uniform(0.0, 50.0, n))
        co2 = np.where(y == 1, np.random.uniform(600.0, 1400.0, n), np.random.uniform(400.0, 520.0, n))
        df = pd.DataFrame({"Temperature": temp, "Humidity": hum, "Light": light, "CO2": co2, "Occupancy": y})

    X = df[FEATURES]
    y = df[TARGET]
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42, stratify=y)
    model = DecisionTreeClassifier(max_depth=5, random_state=42)
    model.fit(X_train, y_train)

    acc = accuracy_score(y_test, model.predict(X_test))
    cm = confusion_matrix(y_test, model.predict(X_test))
    return model, acc, cm

def main():
    st.title("📡 MQTT Room Telemetry & Decision Tree Occupancy Dashboard")
    st.markdown("**CAE 2 Combined Question 3**: MQTT Protocol + Decision Tree Classifier (`paho-mqtt` + `scikit-learn` + `Streamlit`)")
    st.markdown("---")

    model, acc, cm = get_trained_model()

    with st.sidebar:
        st.header("⚙️ MQTT Settings")
        st.write(f"**Broker:** `{BROKER}`")
        st.write(f"**Topic:** `{TOPIC}`")
        st.markdown("---")
        st.header("📊 Model Metrics")
        st.metric("Test Accuracy Score", f"{acc * 100:.2f}%")
        st.subheader("Confusion Matrix")
        st.dataframe(pd.DataFrame(cm, index=["Actual Vacant", "Actual Occupied"],
                                  columns=["Pred Vacant", "Pred Occupied"]))

    if "latest_telemetry" not in st.session_state:
        st.session_state.latest_telemetry = {
            "Temperature": 22.5, "Humidity": 28.0, "Light": 480.0, "CO2": 890.0, "timestamp": time.strftime("%H:%M:%S")
        }

    col_btn, col_info = st.columns([1, 3])
    with col_btn:
        listen_clicked = st.button("🔄 Pull Latest MQTT Telemetry", type="primary", use_container_width=True)
    with col_info:
        st.info(f"Make sure `python publisher.py` is running to send continuous messages to `{TOPIC}`.")

    if listen_clicked:

        received_msg = []
        def quick_msg(client, userdata, msg):
            received_msg.append(msg.payload.decode("utf-8"))

        try:
            temp_client = mqtt.Client()
            temp_client.on_message = quick_msg
            temp_client.connect(BROKER, PORT, keepalive=10)
            temp_client.subscribe(TOPIC)
            temp_client.loop_start()
            t0 = time.time()
            while time.time() - t0 < 3.5 and len(received_msg) == 0:
                time.sleep(0.2)
            temp_client.loop_stop()
            temp_client.disconnect()

            if received_msg:
                st.session_state.latest_telemetry = json.loads(received_msg[-1])
                st.success("Successfully fetched incoming MQTT packet!")
            else:
                st.warning("No new MQTT message arrived in time. Displaying last state.")
        except Exception as e:
            st.error(f"MQTT Error: {e}")

    data = st.session_state.latest_telemetry

    st.subheader("1. Incoming Telemetry Values")
    c1, c2, c3, c4 = st.columns(4)
    c1.metric("Temperature", f"{data['Temperature']} °C")
    c2.metric("Relative Humidity", f"{data['Humidity']} %")
    c3.metric("Light Level", f"{data['Light']} Lux")
    c4.metric("CO₂ Level", f"{data['CO2']} ppm")

    st.subheader("2. Real-Time Occupancy Classification")
    in_vec = [[data["Temperature"], data["Humidity"], data["Light"], data["CO2"]]]
    pred = model.predict(in_vec)[0]
    prob = model.predict_proba(in_vec)[0]

    b1, b2 = st.columns([2, 2])
    with b1:
        if pred == 1:
            st.error("### 🔴 CURRENT STATUS: ROOM OCCUPIED")
            st.write("Occupancy confirmed by high light level & elevated CO₂ readings.")
        else:
            st.success("### 🟢 CURRENT STATUS: ROOM VACANT")
            st.write("Room unoccupied. Energy conservation policies can be activated.")

    with b2:
        st.write("**Classification Confidence:**")
        st.progress(float(prob[1]))
        st.caption(f"Occupied: {prob[1]*100:.1f}% | Vacant: {prob[0]*100:.1f}%")

    st.markdown("---")

    st.subheader("3. Manual Parameter Simulator")
    s1, s2, s3, s4 = st.columns(4)
    st_t = s1.slider("Temperature (°C)", 17.0, 30.0, float(data["Temperature"]))
    st_h = s2.slider("Humidity (%)", 15.0, 50.0, float(data["Humidity"]))
    st_l = s3.slider("Light (Lux)", 0.0, 800.0, float(data["Light"]))
    st_c = s4.slider("CO2 (ppm)", 350.0, 2000.0, float(data["CO2"]))

    sim_res = model.predict([[st_t, st_h, st_l, st_c]])[0]
    st.info(f"Simulated Classification: **{'ROOM OCCUPIED' if sim_res == 1 else 'ROOM VACANT'}**")

if __name__ == "__main__":
    main()
--------------------------------------------------------------------------------
