IJCNC 01

RLIOT: Reinforcement Learning – Based Network Resource Optimization using IoT Sensor Data

Le Hoang Hiep, Van-Tam Hoang and Huu-Huy Ngo, Thai Nguyen University of Information and Communication Technology, Vietnam

 Abstract

The fast rise of wireless communication networks, including 6G, Internet of Things (IoT), and edge com puting, has created unprecedented demand for spectrum and energy resources.become a significant challenge in modern IoT networks due to heterogeneous devices, dynamic traffic patterns, and diverse QoS requirements. This study proposes a Deep Reinforcement Learning (DRL)–based framework for optimizing network resource allocation in IoT environments using real-world sensor data. The proposed framework differs from existing studies that typically assess reinforcement learning methodologies under simplified wireless network assumptions and idealized conditions. Our method functions on heterogeneous IoT traffic produced by various device types, including sensors, actuators, and cameras, each possessing distinct Quality of Service (QoS) requirements. To ensure practical applicability, a realistic IoT simulation environment is developed, incorporating dynamic bandwidth release and queue-aware resource management to emulate real-world network behavior. Furthermore, a Deep Q-Network (DQN) agent with an enhanced exploration strategy is designed to improve learning stability and convergence performance, enabling more efficient and adaptive resource allocation in dynamic IoT scenarios. Experimental results show that the proposed DQN agent achieves a 26.7% improvement in cumulative reward compared to a random policy and consistently outperforms conventional heuristic approaches. This significant gain indicates that the agent effectively learns a structured resource allocation strategy rather than making uninformed decisions. These results confirm that reinforcement learning–based resource allocation provides a scalable and effective solution for IoT networks, particularly in environments characterized by large state spaces, dynamic network conditions, and stochastic traffic patterns.

 Keywords

Deep reinforcement learning, Resource allocation optimization, Internet of Things, Deep Q-Network.

1. INTRODUCTION

The explosive growth of IoT technology in recent years has created unprecedented challenges in network resource management. According to a Cisco report, the number of connected IoT devices is expected to exceed 30 billion by 2030, generating a massive volume of data with diverse and constantly changing QoS requirements. In this context, traditional resource allocation methods [1, 2] based on fixed rules or convex optimization have been widely implemented; however, they often struggle because they cannot adapt to the dynamism and heterogeneity of IoT networks. Also, the dense building of base stations for the IoT system created severe co-tier and cross-tier noise, making the efficient management of spectrum, transmit power, and user association critical for network resources. Optimizing these resources [1] is essential not only to maximize throughput but also to ensure fairness and QoS for all users in the network. Furthermore, these systems must operate under highly complex conditions such as unpredictable user mobility, fast-fading channels, and heterogeneous QoS requirements in the IoT devices. Reinforcement learning (RL) has proven to be a powerful tool for solving sequential decision making problems in uncertain environments [2]. RL enables agents to learn optimal policies through trial-and-error interactions with the environment and eliminates the dependency on fully specified mathematical models [3]. Among RL techniques, DRL has emerged as a powerful framework that integrates deep neural networks to approximate value functions or policies for scaling to high-dimensional and continuous state spaces. Recent studies by Luong [4, 5] and Zhang [6, 7] have shown the potential of DRL in optimizing wireless networks. Additionally, advanced architectures utilizing Reconfigurable Intelligent Surfaces (RIS) paired with DRL have been actively explored to realize intelligent anti-jamming capabilities in emerging 6G communications [8]. However, most of these studies focus on simple wireless networks with assumptions of orthogonal access, perfect channel state information (CSI), and independent channel models. In DQN, in particular, the study of [9] on power allocation in a wireless network using DQN achieved impressive results with a throughput of 3.88 Mbps and a Jain fairness index of 0.91, but still operated in an idealized environment with three homogeneous users and a channel model following a uniform distribution. However, practical environments are inherently non-ideal and deviate from idealized assumptions. This study presents a novel reinforcement learning method in the network resource allocation process for IoT devices using sensor data. The proposed method selectively transmits critical features for sensor transmission tasks, thereby enhancing performance while reducing resource consumption.

The main contributions are presented briefly as follows.

1. We developed an RL-IoT system that uses real-world data from IoT devices to evaluate network resource allocation for diverse device types, i.e., sensors, actuators, and cameras. Their complex characteristics such as packet size, transmission time, latency, jitter, and packet loss rate are also evaluated to demonstrate the efficiency of our method in this resource allocation process.

2. We introduced an efficient multi-objective reward function that not only optimizes throughput but also improves resource allocation accuracy in multiple QoS metrics such as lower bandwidth, latency, packet loss, power, and prioritized device type. We also improved the DQN training process with an epsilon-based attenuation strategy and conducted a detailed convergence analysis on a large-scale IoT dataset.

3. We provided a comprehensive evaluation of the performance of the proposed method against baselines. Our method increased the served rate by 26.7% and achieved a rejection rate of 0.5% while maintaining high stability with a very low standard deviation. The experimental results demonstrate that our method achieves high performance in network resource allocation under real-world IoT environments and devices. Therefore, our system moves closer to real-world IoT network characteristics, including dynamic bandwidth allocation mechanisms based on random service times and limited-size request queues.

Unlike previous studies that mainly evaluated DRL methods in simplified wireless environments with homogeneous users, the proposed RL-IoT framework operates on real-world heterogeneous IoT traffic data and integrates dynamic queue management, probabilistic bandwidth release, and multi-objective QoS-aware rewards. These characteristics provide a more realistic evaluation environment and improve the robustness of the learned allocation policy. The remainder of the study is organized as follows. Section 2 discusses the related works. Section 3 describes the proposed reinforcement system model. Section 4 presents the obtained performance of the proposed system’s network resource allocation. Section 5 concludes the paper and presents future work.

2. RELATEDWORK

Network resource optimization has been extensively studied in wireless and IoT systems using both classical optimization and DRL approaches. Traditional methods typically rely on convex optimization, heuristic scheduling, or rule-based allocation strategies. While these methods pro vide theoretical guarantees under simplified assumptions, they often struggle to adapt to highly dynamic and heterogeneous environments such as large-scale IoT networks with diverse sensor de vices and time-varying traffic patterns. DRL has emerged as a powerful alternative for sequential decision-making problems in networking due to its ability to learn adaptive policies directly from interaction data. Foundational work on DRL demonstrated that deep neural networks can approx imate value functions and policies in large state spaces, enabling practical solutions for complex control problems [10, 11]. Building on these advances, several studies have highlighted the grow ing role of DRL in wireless communications and networking, including routing, spectrum access, and resource allocation [4, 12, 13]. These studies show that DRL-based approaches frequently outperform static and heuristic baselines in dynamic scenarios. Also, these approaches typically design reward functions that jointly capture throughput, fairness, delay, or nergy cost, enabling policy learning that balances competing QoS metrics [14]. However, these existing studies rely on simplified simulation settings with homogeneous users, stationary traffic models, or synthetic channel distributions. As a result, the learned policies may not apply to realworld IoT traffic due to different among them such as noisy sensor data, busty arrivals, and mixed device types. Additional, they did not consider into the density of the IoT devices or the capability to distribute data to multiple devices simultaneously. Other works investigate distributed and multi-agent DRL architectures for scalable resource management in edge and fog computing environments, show ing improved coordination and scalability compared to single-agent designs [15]. These studies indicate that DRL is a promising direction for practical network optimization, but also highlight open challenges in stability, sample efficiency, and real-world deployment.

In the IoT domain, network resource management presents significant challenges due to device heterogeneity, QoS diversity, and constrained resources. Recent studies specifically focusing on reinforcement learning for IoT and edge/fog systems report that DRL can effectively support adaptive resource provisioning, task offloading, and bandwidth allocation under uncertainty [17,18]. Therefore, DRL-based approaches have been applied to computation offloading and joint communication-computation resource allocation, achieving improved delay, energy efficiency, and system utility compared to conventional approaches [19]. Additionally, energy-aware routing strategies for low-power and lossy IoT networks have been investigated to reduce energy consumption while maintaining reliable data transmission [20], highlighting the importance of energy efficiency as a key optimization objective alongside throughput and latency. However, network resource allocation in these studies has not yet been considered in real-world environments with heterogeneous IoT devices. In contrast to prior work, which mainly evaluates DRL methods in idealized or small-scale environments, this paper focuses on DRL-based network resource optimization using real sensor-driven IoT traffic data within a dynamic environment incorporating stochastic resource release and queue dynamics. This setting better reflects practical IoT network behavior and enables a more realistic assessment of allocation policy robustness and stability in future RL-IoT systems.

3. PROPOSAL METHOD

This section presents the proposed RLIoT method to the reinforcement learning-based network resource optimization for IoT devices via two parts. The details of these two parts are presented in the following subsections.

3.1. MDP-Based Network Resource Allocation
Network resource allocation in IoT environments is increasingly challenging due to the rapid growth of connected devices. Therefore, we develop the Markov Decision Process(MDP)- ased system to optimize recourse for dense IoT devices, a standard mathematical abstraction for sequential decision making problems under uncertainty. This MDP framework allows an agent to interact with a stochastic environment over discrete time steps, learning to make decisions that maximize expected long-term rewards. My target is aiming to develop a deep MDP to control these resource allocation that is defined by the 5-tuples as in Equation 1.The objective of the MDP is to compute an optimal states for the IoT systems.

Where: (1) State Space (S): Let a observed state st∈ S is a measurable space representing all possible system states in each element st∈ S and is identified by

In our system, a sate st∈R
13 is designed by two key parts including: (1) the nine feature vectors
i.e., xsize,xtime,xbw, xlat,xjit,xloss,xenergy,xproto,xdevice has been normalized to mean 0 and standard deviation 1 using Standard Scaler [21]. Also, the classification features i.e., protocol and device types has encoded using Label Encoder[21] to suitable with proposed system.(2) We build the network state vectors with 4-elements ubw, uqueue, uutil, utime, that are computed by ubw= Balloc/Bmax, uqueue= |Q|/Qmax, utime= t/T, where, ubwis the bandwidth using rate with max =100Mbps,uqueueisthequeuefullratewithQmax=20,uutil is signal enhancement by repeating ubw, and utime is the time progress in the episode.

 2) Action Space(A): A denotes the set of admissible actions for a state sss, and each sss can select an action in A due to st∈A(st) to determine the next decision process. Therefore, in this setting corresponds to the transmit power configuration for all users is presented by

where each pi(t)p_i(t)pi(t) denotes the power allocated to user iii at time ttt. In our system, we fix the set A={0,1,2,3,4}A = \{0,1,2,3,4\}A={0,1,2,3,4} to correspond to the bandwidth allocation levels as follows. When the state sts_tst selects action A=0A = 0A=0, it represents a denial of service, and the request is placed in a queue if there is available space. Similarly, the action set A={1,2,3,4}A = \{1,2,3,4\}A={1,2,3,4} represents bandwidth allocation levels of 20%, 40%, 60%, and 80%, respectively. We select discrete allocation levels to reflect practical hardware limitations and to reduce the complexity of the action space compared with continuous allocation in real-world environments.

 3)TransitionModelP(·|s,a):WepresentthetransitionkernelP.Itisastochastickernel,i.e., P(·|s,a) defined on S by

For this system, the distribution process for each user is identified through two key mechanisms. First, the bandwidth allocation mechanism thoroughly checks the action corresponding to the request. If balloc= breq×αa≤ Bmax−Balloc, the system performs the allocation balloca nd adds it to the list of active allocations Lactive, otherwise, the request is placed into the queue Q provided that|Q|≤Qmax. If the queue is full, the request is rejected entirely. Second, the bandwidth release mechanism is deployed. The probability function iterates through All active allocations(bi,ti)∈Lactive and computes the completion probability by pcomplete= min(0.3 + 0.01 ×(t −ti) ×0.8). Therefore, Once the system obtains pcomplete, it releases the bandwidth ti and also removes the function Lactive that contains ti. This mechanism reflects real-world conditions in which IoT connections experience random service times, and network bandwidth is gradually released over time.
(4) Reward Function rt∈R : Based on the specific requirements in our system. We are designed an efficient reward function to reflect the multi-objective computation of real-world IoT problems as in Equation 5. While previous research focused solely on output and cost performance, our rewards encompass multiple components.

The weights are set as w1 = 3.0,w2 = 2.0,w3 = 2.0,w4 = 1.0,w5 = 1.0, balancing allocation accuracy (most important), QoS metrics, and energy efficiency. This diversity reflects the reality of IoT networks with devices having heterogeneous QoS requirements, from low-power sensors to highbandwidth cameras. This function will assign an immediate reward to each state-action pair. However, due to Iot devices variety, we designed this function with many components to reflect different objectives. Regarding the act of refusing service i.e., at = 0, the reward is rt = −2. If the request is successfully added to the queue, and rt = −5 if the queue is full. Similarly, for allocation actions (at ∈ {1,2,3,4}), if sufficient bandwidth is available, the reward is computed using the aggregate formula described in the previous section, balancing resource allocation and accuracy.
(5) Discount Factor (γ): In our system, we are using the exponential decay function to heavily penalize large allocation deviations. To obtain the efficient reward function, we use the discount factor γ ∈ [0,1) which determines how future rewards are weighted relative to immediate ones. We choose a value close to 1 i.e., γ = 0.95 which encourages the agent to value long-term performance, fostering stable and proactive behavior. Our simulation environment is also significantly more complex. Instead of a simple channel model, we implement a dynamic bandwidth release mechanism based on random service time. Each bandwidth allocation has a completion probability pcomplete that depends on time. This mechanism simulates the reality that IoT requests have different service times and bandwidth is released in real time, not instantaneously as in idealized models. Furthermore, we implement a request queue with a maximum size of 20, forcing the agent to learn how to manage congestion and avoid rejecting requests when the queue is full. This mechanism helps reduce packet loss by temporarily buffering requests when bandwidth insufficient. Because when the current band width is insufficient to handle a new request, instead of immediately rejecting it i.e., causing data loss, the system will place the request in a processing queue.

3.2. DQN-Based Resource Allocation Agent
While previous studies employ a DQN architecture that uses a two-layer hidden neural network with 64 and 128 neurons, respectively, and ReLU activation functions. Our DQN agent is further enhanced by a13-dimensional states pace, consisting of nine network features and four system state variables: bandwidth utilization rate, queue rate, utilization level, and time progress. The neural network architecture comprises three hidden layers with dimension of 128, 128, 64, and employs Batch Normalization along with Dropout (rate = 0.2) to enhance generalization performance. More importantly, we use an episode-based epsilon decay strategy with a rate of 0.995 instead of step-based decay as in many previous studies. This ensures that the agent has enough time to explore the state-action space before committing to a specific policy, leading to more stable convergence. To obtain the most efficient action function for our system. We develop action-value function under policy π that is defined by Equation 6 and the optimal action-value function satisfies the Bellman optimality equation that is computed 

Our research significantly extends previous works in several important directions. While previous studies relied on aggregated data drawn from a uniform distribution, we utilize real-world IoT network data comprising 100,000 traffic samples collected from a diverse range of devices. Our dataset contains nine key features: packet size, transmission time, bandwidth usage, latency, jitter, packet loss rate, energy consumption, protocol type, and device type. These variations are designed to reflect real network environments with dense IoT devices. To ensure a stable learning process within the system, we employ two key techniques, as follows.(1) Experience replay with a buffer of size10,000 stores transition tuples(st,at,rt,st+1,dt) and random mini-batches of size 64 are sampled to break temporal correlations in the training data.(2) Target Network Q(s,a,θ− )is a replica of the main network updated every 10 episodes by copying the weights(θ −← θ), helping to stabilize the learning target. The learning algorithm employs an ϵ-greedy strategy where the action is selected according to at=arg maxa Q(st,a,θ) with probability 1-ϵ(exploitation)or selected randomly with probability ϵ(exploration). The loss function is the Mean Squared Error (MSE) between the predicted Q-value and the target Q-value, which is computed by A key improvement is the epsilon decay strategy. Un like previous studies which typically decayed epsilon after each step, we decayed after each episode computed by ϵ← max(ϵ×0.995,ϵmin), with ϵstart= 1.0 and ϵmin = 0.01.Our DQN model is optimized using Adam optimizer with learning rate of α = 0.0005 and discount factor of γ= 0.95.These settings allow the agent sufficient time to explore diverse state–action configuration sin each episode before gradually shifting toward exploitation, resulting in more stable convergence and an improved final policy.

3.3. Network Resource Allocation Framework
Fig. 1 illustrates the DQN architecture for the RL-based IoT bandwidth allocation problem, characterized by a 13-dimensional state space and a discrete action space with five allocation levels. The agent employs a Deep Q-Network (DQN) with three hidden layers, together with an experience replay buffer, and a target network, to learn the optimal policy mapping. The learning objective is to maximize a multi-objective reward function defined in terms of allocation error, QoS metrics, and energy efficiency. This allocation processing presents as in Algorithm1. The algorithm operates in an episode–step loop with three main components.

Figure 1. Overall architecture of the DQN-based system for IoT network resource allocation. IoTDL observes the state st, and the DQN agent selects an action to allocate resources for the execution phase e.g., either 0%, or 20%, 40%, etc. This process repeats and stabilizes the learning procedure, thereby facilitating optimal resource allocation.

As a result, our RLIoT model gradually shifts from exploration to exploitation. During this process, the target network is synchronized with the online network every 10 episodes θ −← θ to stabilize the training process. This design ensures adequate exploration within each episode before progressively increasing exploitation, there by promoting more stable convergence in during training. In contrast to step-based decay strategies adopted in prior studies, the proposed approach implements epsilon decay on a per-episode basis.

4. EXPERIMENTALRESULTS
This section details the evaluation results of the allocation performance on the proposal method that is presented in Section 3.The dataset includes traffic generated from three types of realIoT devices, namely sensors, actuators, and cameras, and provides comprehensive features such as packet size, latency, jitter, energy usage, and other related network attributes. The detail is presented in subsection as follows.

Figure 2. The DQN training process was conducted over 200 episodes. In the top-left image, the convergence process shows that the reward increased from approximately 6,500 to 7,400. In the top-middle image, the loss remained near zero for most of the training session. In the top-right image, the decay rate was set to 0.995 per episode. The number of served requests increased from approximately 750 to 870, while the number of rejected requests decreased from approximately 250 to 130, as shown in the bottomleft and bottom-middle images, respectively. As a result, the proposed RLIoT framework achieved significant performance improvements

4.1. Experimental Setup
All experiments were conducted on a Tesla P100 GPU using Tensor Flow 2.19.0 and Python 3.10. We use the IoT Network Traffic dataset 1 that consists of 100,000 samples and we were divided into a training set with first 80% samples and a testing set with 20% samples. We compared the DQN agent with three baseline policies: (i) a Random Policy, which selects actions randomly from {0,1,2,3,4}; (ii) a Greedy Policy, which always selects action 4 (corresponding to the maximum allocation of 80%); and (iii) a Conservative Policy, which always selects action 1 (corresponding to the minimum allocation of 20%). Each policy was evaluated over 20 independent episodes on the test set to compute the mean and standard deviation of the performance metrics.

4.2. Learning Convergence and Stability Process
The training results indicate that the DQN agent converges stably after approximately 150 180
episodes as descried in Fig. 2. The convergence behavior is demonstrated by the steady increase in cumulative reward increasing about 20% compared to the study [9]. Our experimental setup employs a different training configuration, utilizing 200 episodes in different environments. The loss remains stable for most of the training process, followed by a sudden spike around episode 170. This shows that the agent is still able to adapt to difficult test cases but recovers quickly, demonstrating its robustness.

Figure 3. Detailed of the DQN learning Process. The change of network resource under the episodes during training in our system.

diverse devices, episode-based decay ensures agents have enough time to fully explore all scenarios in each episode before reducing exploration. If decay is done step-by-step, like in [9], agents might be forced into exploitation too early, before seeing enough cases in a long episode, leading to learning suboptimal policies. The service success ratio remains relatively stable at around 0.85 throughout the training process, with a noticeable spike near episode 170. This fluctuation corresponds to the anomaly previously observed in the reward curve, further confirming the relationship between performance metrics and learning dynamics. In short, the average reward increases progressively from about 6,730 in the first 10 episodes to approximately 7,000–7,200 around episodes 100–130, and further improves to about 7,400 in the final stage when the agent refines its policy with a lower epsilon value. The epsilon parameter decays from 1.0 to 0.670 after 200 episodes according to the designed schedule, ensuring a proper balance between exploration and exploitation. The average training loss gradually decreases over time, demonstrating that the neural network learns a better representation of the Q-function. The number of served requests increases from 798 to 864 per episode, while the number of rejected requests decreases from 200 to 135, indicating that the agent adapts effectively to varying network conditions.

4.3. DQN- Based Distribution Performance
detailed analysis as in Fig. 3 reveals a notable phenomenon at episode 170, where the reward drops sharply because the agent encounters a difficult test case while the epsilon value remains at 0.32 (i.e., 32% exploration). Similar to Fig. 2 (DQN training curve) in [9], but we extended the figure by adding a1 standard deviation band (shaded region) to reflect training stability as described in the top-left image. The progressively narrowing standard deviation band indicates that the learned policy becomes more Figure 3: Detailed of the DQN learning Process. The change of network resource under the episodes during training in our system. stable over time. For episode, RLIoT improves rate fluctuates around 0%, with a prominent spike at approximately episode 170, reaching about 18% improvement after recovery from the observed anomaly as in the top-right image.

4.4. Network Resource Allocation Performance
Evaluation on the test set shows that the DQN agent achieves superior performance as presented in Fig. 4. In terms of cumulative reward, the DQN achieves 8685.60 ± 0.52, significantly outper forming the Random Policy (6853.76±117.00), with a relative improvement of 26.7%. Compared with the Greedy Policy (8355.67±0.0), DQN achieves a 3.95% gain, demonstrating that an adap tive learned policy performs better than always selecting the maximum allocation. Notably, the Conservative Policy yields the highest cumulative reward (8941.17 ± 0.0), exceeding DQN by 2.9%. The Conservative Policy can achieve higher cumulative rewards than DQN because it prioritizes stability, minimizes exploration risk, and avoids convergence and estimation issues while our DQN prioritizes low-bandwidth requests, where a conservative strategy (20% allocation) avoids penalties caused by over-allocation.

Figure 4. Performance comparison off our policies on the test set

4.5.Analysis and Discussion
The experimental results demonstrate the effectiveness of the DRL-based approach in address ing complex IoT resource allocation problems. The DQN agent achieves a 26.7% improvement over baseline policies, indicating its ability to learn an adaptive strategy tailored to the specific characteristics of each request and the current network state. This property is particularly im portant in heterogeneous IoT environments, where devices have diverse QoS requirements. The proposed multi-objective reward function successfully guides the agent to balance multiple objec tives, including allocation accuracy close to the optimal bandwidth, QoS preservation (low delay and packet loss), energy efficiency, and priority support for critical devices. The superior performance of the Conservative Policy (8,941.17) compared to DQN (8,685.60) suggests that the current reward function may require further refinement to better balance allocation accuracy and efficiency.

Table 1. Comparison Between the Proposed RL IoT Framework and Existing DRL-based Resource Allocation Studies

of multiple QoS aspects. The proposed environment compared with frame work is evalu prior DRL based stud ated in a more realistic and dynamic IoT ies. While their approach reports a throughput of 3.88 Mbps and a fairness index of 0.91 in a simplified simulation involving three homogeneous users, our framework processes real-world data comprising hundreds of heterogeneous devices per episode and give outcome results. This is a promising method with multi-devices in real environment also.

5. CONCLUSION AND FUTURE WORK
This study proposes a DRL-based framework for resource allocation optimization in IoT networks using real-world traffic data. Our approach operates directly on heterogeneous IoT traffic with diverse device types and non-uniform QoS requirements. Our RLIoT evaluated on a realistic IoT environment model with dynamic bandwidth release and queue management, a multi-objective re ward function balancing allocation accuracy and multiple QoS metrics, and an improved training strategy with episode-based epsilon decay. Experimental results demonstrate that the DQN agent achieves a 26.7% improvement in cumulative reward over the Random policy and 3.95% over the Greedy policy, while maintaining high stability with very low standard deviation. Our method substantially extends DRL applicability to more complex and realistic IoT settings with hetero geneous data and multi-dimensional reward design. Future research directions include extending the framework to multi-agent DRL architectures for large-scale IoT systems with thousands of dis tributed devices, where each gateway or base station operates as an independent cooperative agent. Continual learning mechanisms should be incorporated to enable adaptation to time-varying traf fic distributions without retraining from scratch, which is particularly important for IoT workloads that vary by time of day or season. Further validation should be conducted on real IoT testbeds or high-fidelity simulators such as NS-3 to evaluate performance under inter-device interference, packet loss, and communication delays. Explainable AI techniques should also be investigated to interpret agent decisions and increase deployment trustworthiness in critical systems. Finally, hybrid architectures combining DRL with traditional optimization methods represent a promising direction, in which DRL handles complex scenarios while classical optimization addresses sim pler cases to ensure baseline performance and reduce computational costs.

CONFLICTS OF INTEREST
The authors declare no conflicts of interest.

REFERENCES
[1] O. Giwa, J. Shock, J. D. Toit, and T. Awodumila, “Optimisation of resource allocation in heterogeneous wireless networks using deep reinforcement learning,” arXiv preprint arXiv:2509.25284, 2025.
[2] H. Jiang, T. Wang, and S. Wang, “Multi-Agent Reinforcement Learning for Dynamic Spectrum Access,” in Proceedings of the IEEE International Conference on Communications (ICC). IEEE, 2019, pp. 1–6.
[3] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA: MITPress, 2018.
[4] N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P. Wang, Y.-C. Liang, and D. I. Kim, “Applications of Deep Reinforcement Learning in Communications and Networking: A Survey,” IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3133–3174, 2019.
[5] Y. Xu, Z. Zhao, P. Cheng, Z. Chen, M. Ding, B. Vucetic, and Y. Li, “Constrained Reinforcement Learning for Resource Allocation in Network Slicing,” IEEE Communications Letters, vol. 25, no. 5, pp. 1554–1558, 2021.
[6] L. Huang, S. Bi, and Y.-J. A. Zhang, “Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks,” IEEE Transactions on Mobile Computing, vol. 19, no. 11, pp. 2581–2593, 2020.
[7] F. G. Ortiz-Gomez, R. Martinez, M. A. Salas-Natera, and A. Garcia-Perez, “Cooperative Multi-Agent Deep Reinforcement Learning for Resource Management in Full Flexible VHTS Systems,” IEEE Transactions on Cognitive Communications and Networking, vol. 8, no. 1, pp. 335–349, 2021.
[8] L. H. Hiep, H. M. Viet, and T. D. Minh, “RIS assisted Deep Reinforcement Learning for intelligent Anti-Jamming communications in 6G Networks,” TNU Journal of Science and Technology, vol. 231, no. 06, pp. 419–428, 2026.
[9] M. D. Iradukunda, C. F. Elégbédé, and Y. U. Gaba, “Intelligent resource allocation in wireless networks via deep reinforcement learning,” arXiv preprint, arXiv:2601.04842v2, Jul. 2026.
[10] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015.
[11] F. Hussain, R. Hussain, S. A. Hassan, and E. Hossain, “Machine learning for resource man agement in cellular and iot networks: Potentials, current solutions, and open challenges,” IEEE Communications Surveys & Tutorials, vol. 22, no. 2, pp. 1251–1275, 2020.
[12] P.-Y. Chou, W.-Y. Chen, C.-Y. Wang, R.-H. Hwang, and W.-T. Chen, “Pricing-Based Deep Reinforcement Learning for Live Video Streaming With Joint User Association and Resource Management in Mobile Edge Computing,” IEEE Transactions on Wireless Communications, vol. 21, no. 6, pp. 4310–4324, 2022.
[13] M.Alsenwi, N. H. Tran, M. N. H.Nguyen, S. R. Pandey, K. S. Kim, and C. S. Hong, “Intelligent resource slicing for eMBB and URLLC coexistence in 5G and beyond: A deep reinforcement learning based approach,” IEEE Transactions on Wireless Communications, vol. 20, no. 7, pp. 4585–4600, 2021.
[14] Q. Wu, Z. Zhang, H. Zhu, P. Fan, Q. Fan, H. Zhu, and J. Wang, “Deep Reinforcement Learning-Based Power Allocation for Minimizing Age of Information and Energy Consumption in Multi-Input MultiOutput and Non-Orthogonal Multiple Access Internet of Things Systems,” Sensors, vol. 23, no. 24, p. 9687, 2023.
[15] D. Tse and P. Viswanath, Fundamentals of Wireless Communication. Cambridge, UK: Cambridge University Press, 2005.
[16] R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforce ment learning,” Machine Learning, vol. 8, no. 3, pp. 229–256, 1992.
[17] D. Hortelano, I. de Miguel, R. J. Durán Barroso, J. C. Aguado, N. Merayo, L. Ruiz, A. Asensio, X. Masip-Bruin, P. Fernández, R. M. Lorenzo, and E. J. Abril, “A comprehensive survey on reinforcement-learning-based computation offloading techniques in Edge Computing Systems,”journal of Network and Computer Applications, vol. 216, p. 103669, 2023.
[18] H. Tran-Dang, S. Bhardwaj, T. Rahim, A. Musaddiq, and D.-S. Kim, “Reinforcement learning based resource management for fog computing environment: Literature review, challenges, and open issues,” Journal of Communications and Networks, vol. 24, no. 1, pp. 83–98, 2022.
[19] X. Qiu, W. Zhang, W. Chen, and Z. Zheng, “Distributed and collective deep reinforcement learning for computation offloading: A practical perspective,” IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 5, pp. 1085–1101, 2021.
[20] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P.
Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and É. Duchesnay, “Scikit-learn: Machine Learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011. [Online]. Available: https://jmlr.org/papers/v12/pedregosa11a.html
[21] Y. Cao, S.-Y. Lien, and Y.-C. Liang, “Deep reinforcement learning for multi-user access control in non-terrestrial networks,” IEEE Transactions on Communications, vol. 69, no. 3, pp. 1605–1619, 2021.

AUTHORS
Le Hoang Hiep received the B.E. degree in Information Technology from Thai Nguyen University of Information and Communication Technology (ICTU), Vietnam, in 2009, and the M.S. degree from Manuel S. Enverga University Foundation, Philippines, in 2013. He is currently pursuing the Ph.D. degree at ICTU, Vietnam. His research interests include wireless network security, deep reinforcement learning, the Internet of Things (IoT), and UAV–satellite communications.

Van-Tam Hoang (Corresponding author) received his Ph.D. in the Department of Computer Science and Information Engineering, National Chung Cheng University, Taiwan, in 2025. He is currently a lecturer in the Faculty of Information Technology at the University of Information and Communication Tech nology, Thai Nguyen, Vietnam. His research interests include security, deep learning, IoT, embedded system, quantum, and semantic communications

Huu-Huy Ngo received his B.S. and M.S. degrees from Thai Nguyen University of Information and Communication Technology, Vietnam, in 2010 and 2012, respectively, and Ph.D. degrees in Information Engineering and Computer Science from Feng Chia University, Taiwan, in 2021. Currently, he is a lecturer at the Thai Nguyen University of Information and Communication Technology, Vietnam. His research interests include computer vision, deep learning, deep reinforcement learning, Internet of Things (IoT), and neural networks.