IJCNC 03

MULTI-AGENT REINFORCEMENT LEARNING FOR TASK OFFLOADING AND RESOURCE ALLOCATION IN MEC SYSTEMS

Youssef Oukissou¹ , Mohamed Amine Meddaoui² , Ayoub Belaidi³ and Mohamed Oubezza⁴

¹ Information Processing and Decision Support Laboratory, University of Sultan Moulay Slimane, Faculty of Science and Technologies, Beni Mellal 23500, Morocco

²Data4Earth Laboratory, Sultan Moulay Slimane University, Beni Mellal, Morocco

³ Information Processing and Decision Support Laboratory, University of Sultan Moulay Slimane, Faculty of Science and Technologies, Beni Mellal 23500, Morocco

⁴Department of Computer Science, Faculty of Science, Chouaib Doukkali University, El Jadida, Morocco

ABSTRACT

Multi-Access Edge Computing (MEC) brings computation closer to end users to reduce latency and energy consumption for compute-intensive mobile applications. In this paper, we address the joint task offloading and resource allocation problem in multi-user MEC systems and propose a decentralized control framework based on Multi-Agent Reinforcement Learning (MARL). Each user–base station link is modeled as an autonomous agent that decides whether to execute tasks locally or offload them to the edge, and how much computing resource to request. The learning framework follows a centralized training and decentralized execution paradigm and integrates Q-Learning, Deep Q-Network (DQN), and Double DQN (DDQN) algorithms. Extensive simulations show that the DDQN-based approach achieves lower total system cost and faster convergence than the full-local, full-offload, and heuristic baselines. The results confirm that lightweight MARL is a practical and scalable solution for dynamic MEC environments under realistic resource constraints.

KEYWORDS

Multi-Access Edge Computing, Task Offloading, Resource Allocation, Multi-Agent Reinforcement Learning, Decentralized Learning.

1. INTRODUCTION

The rapid proliferation of smart devices and data-intensive mobile applications has led to a fundamental shift in wireless communication networks. Advancements in mobile technology now allow users to perform compute-intensive tasks remotely; however, mobile devices remain constrained by limited processing power and storage capacity [1]. As a result, emerging services such as real-time gaming, augmented reality (AR), and high-definition video streaming are increasingly difficult to support on mobile platforms alone.

Multi-Access Edge Computing (MEC) has emerged as a promising paradigm to address these limitations by extending cloud computing capabilities to the edge of the network [2]. By bringing computation closer to end-users, MEC enables decentralized data processing and significantly reduces latency, network congestion, and energy consumption. Nevertheless, efficient resource

management in MEC remains a critical challenge, particularly in dynamic and heterogeneous environments that involve Internet of Things (IoT) applications that generate large volumes of latency-sensitive data. Resource allocation strategies must therefore balance competing objectives such as execution delay, energy efficiency, and fairness [3–5].

Although traditional cloud computing platforms provide task offloading capabilities [6], they often suffer from high latency and centralized bottlenecks. In contrast, MEC servers deployed on the network edge offer localized processing and improved Quality of Service (QoS) [7], making them well suited for time-critical and bandwidth-intensive applications such as energy-efficient computational offloading (EECO) [8] and fiber–wireless (Fi-Wi) networks [9]. However, the increasing number of connected devices and heterogeneous workloads places significant pressure on both communication and computing resources at the edge.

Multi-Agent Reinforcement Learning (MARL) provides a scalable, decentralized alternative to traditional optimization by enabling autonomous coordination and adaptive resource management in complex, dynamic MEC environments.The main contributions of this paper can be summarized as:followsthis paper introduces a decentralized Multi-Agent Reinforcement Learning (MARL) framework utilizing a Centralized Training and Decentralized Execution (CTDE) paradigm to optimize the trade-off between latency and energy consumption in multiuser MEC systems, demonstrating through extensive simulations that a Double DQN (DDQN) approach significantly outperforms traditional Q-Learning and DQN baselines in both cost efficiency and convergence stability

The remainder of the paper is structured as follows: Section 2 reviews related work. Section 3 introduces the MEC architecture and the system model. Section 4 formulates the optimization problem. Section 5 presents the MARL-based learning framework and reinforcement learning techniques. Section 6 reports simulation results and Section 7 conclude the paper and outlines directions for future work.

2. RELATED WORK

Resource allocation and task offloading in Multi-Access Edge Computing (MEC) systems have been the subject of research in recent years, by offering solutions ranging from traditional heuristic methods to advanced machine learning-based approaches. Traditional heuristics such as greedy scheduling, earliest deadline first (EDF), and round-robin policies are widely adopted due to their low computational complexity and ease of implementation [10–12]. More recent hybrid scheduling strategies have also been proposed to improve reliability and sustainability in cloudbased services [32]. However, these methods are typically static and do not adapt effectively to the highly dynamic nature of MEC environments, where user mobility, task heterogeneity, and fluctuating wireless conditions are prevalent.

To address these limitations, machine learning techniques, and in particular reinforcement learning (RL), have been explored to enable adaptive and autonomous decision-making for resource allocation and task offloading. Early studies investigated tabular Q-learning approaches in centralized MEC settings, demonstrating the potential of RL for dynamic optimization, but suffering from scalability limitations due to large state-action spaces and the need for global state information [5]. As the complexity of the system increases, such methods become impractical for large-scale deployments.

Deep reinforcement learning (DRL) techniques, including Deep Q-Networks (DQN) and Double DQN (DDQN), have been proposed to overcome these challenges by leveraging neural networks for value function approximation [13, 14]. Recent studies in IJCNC have further demonstrated the effectiveness of DRL for resource allocation in next-generation wireless networks, including massive MIMO-NOMA systems [33]. These approaches can handle high-dimensional state spaces and improve generalization between varying task profiles. Nevertheless, most existing DRL-based MEC solutions rely on centralized architectures and assume full observability of the state of the system, which limits their applicability in distributed and resource-constrained edge environments [15].

More recently, Multi-Agent Reinforcement Learning (MARL) has emerged as a promising paradigm for distributed resource management in MEC systems. Under the Centralized Training and Decentralized Execution (CTDE) framework, multiple agents—typically representing users or user–base station links—learn local policies while benefiting from coordinated training [16]. Several variants of MARL have been investigated, including actor–critic-based methods [17], value decomposition techniques for cooperative learning [16], and federated reinforcement learning frameworks designed to reduce communication overhead and preserve data privacy [18]. In parallel, related work published in IJCNC has investigated adaptive load-balancing techniques for resource allocation in fog–cloud environments [34] and improved reinforcement-learningbased routing for wireless networks [35], complementing the present study by addressing closely related distributed resource management challenges.

Despite these advances, significant challenges remain. Many existing MARL-based solutions rely on strong assumptions, such as perfect cooperation, global reward sharing, or full network knowledge, which may not hold in realistic MEC scenarios. Furthermore, empirical evaluations often overlook practical constraints such as partial observability, user mobility, and energy limitations, leading to optimistic performance estimates [19]. The non-stationarity induced by simultaneously learning agents also remains a critical issue in dynamic MEC environments.

In this context, the present work proposes a decentralized MARL framework that models independent agents with local observations and energy-aware decision-making. Unlike fully centralized or coordination-intensive approaches, the proposed method emphasizes lightweight learning mechanisms suitable for edge-centric MEC deployments. To better position this contribution within the existing literature, Table 1 provides a comparative analysis of representative reinforcement learning approaches for resource allocation and task offloading in MEC systems. As shown in the table, traditional DRL scheduling methods rely on centralized control and suffer from high computational overhead [13], while cooperative MARL approaches often require strong coordination mechanisms [16]. Actor-critic-basedsolutions also typically assume full observability, which limits their practicality in distributed environments [17]. In contrast, the proposed CTDE-based MARL framework leverages decentralized execution with DDQN, reducing system cost while maintaining effective performance in heterogeneous and dynamic MEC scenarios.

Figure 1. Comparative Analysis of Reinforcement Learning Approaches for Resource Allocation and MEC Offloading

3. MULTI-ACCESS EDGE COMPUTING

Multi-Access Edge Computing (MEC) is a network paradigm that extends cloud computing capabilities to the edge of the network, offering localized processing for mobile users [20]. By deploying computational resources closer to end-user devices, MEC significantly reduces latency and network congestion, enabling the real-time execution of compute-intensive and latencysensitive applications.

3.1. Multi-Access Edge Computing Architecture

The MEC architecture is typically organized into a three-layer hierarchy consisting of the user layer, edge layer, and cloud layer, as illustrated in Figure 1. This structure clarifies the distribution of roles and interactions within the system. The user layer comprises mobile devices and IoT nodes that generate and consume data, such as sensors, smartphones, vehicles, and smart meters. These devices rely on wireless communication technologies to connect to the system and offload computation-intensive tasks. The edge layer consists of MEC servers or edge nodes deployed at base stations or access points. It provides intermediate computing resources to process tasks offloaded from user devices, thereby reducing latency and improving responsiveness. The cloud layer represents centralized data centers with large-scale computing and storage capabilities, primarily used for delay-tolerant or computation-heavy workloads. Overall, this layered architecture enables flexible task distribution according to latency requirements, energy constraints, and computational demands. Communication between mobile devices and the network infrastructure can be categorized into three modes: infrastructure-based mode, where devices communicate through centralized entities such as base stations or access points; ad hoc mode, where devices communicate directly in a peer-to-peer manner without centralized coordination; and hybrid mode, which combines both approaches and allows dynamic switching based on network conditions and connectivity.

Figure 1. MEC Hierarchical Architecture Offloading

3.2. Challenges in Multi-Access Edge Computing

As with any emerging paradigm, Multi-Access Edge Computing (MEC) systems face several practical and technical challenges that must be addressed to fully realize their potential [22]. These challenges span communication delays, dynamic offloading decisions, and resource allocation constraints.

3.2.1. Communication Delay

While local devices are generally capable of predicting service delays through deterministic models [23], the actual latency experienced in MEC systems is often influenced by the stochastic nature of the wireless environment. For instance, unfavorable channel conditions or high interference levels can significantly increase communication delays, making it difficult to meet stringent real-time constraints. Therefore, accurate and adaptive allocation of physical layer resources such as transmission power and sub channel bandwidth is essential to mitigate latency in edge networks [21].

3.2.2. Offloading Decision Optimization

Although MEC provides opportunities for task offloading and faster execution, offloading is not always the most efficient choice. In scenarios where the wireless link quality is poor or when the transmission time exceeds the local processing time, it may be more effective to process tasks locally or redirect them to alternative servers [24]. This highlights the need for context-aware offloading strategies that adapt dynamically to channel conditions and workload characteristics to ensure minimal latency and optimal resource use.

3.2.3. Computing Resource Allocation

Even when a task is successfully offloaded, the device must often wait for the MEC server to allocate sufficient computational resources for processing. Since edge servers typically have limited processing capabilities compared to centralized cloud infrastructures, the allocation of computational resources becomes a critical factor [25]. Efficient task scheduling and resource management are therefore vital to maintain service quality, reduce processing delays, and serve multiple users concurrently in dense MEC deployments.

3.3. The Resource Allocation Problem in MEC

Efficient resource allocation is a critical aspect of Multi-Access Edge Computing (MEC), aiming to optimize the utilization of limited computational and communication resources while meeting the performance requirements of diverse applications and services. However, this task is inherently challenging due to the highly dynamic nature of MEC environments, characterized by variable workloads, stringent latency constraints, and limited server capacity [26].

Traditional solutions, such as rule-based heuristics or static optimization algorithms, have been widely adopted for their simplicity and low overhead. Nonetheless, these methods often struggle to adapt to fluctuating network conditions and unpredictable user demands, which can result in poor resource utilization and degraded service quality. To overcome these limitations, recent research has focused on more adaptive and intelligent approaches. Among them, Reinforcement Learning (RL) has gained significant attention as a promising solution. As a subfield of artificial intelligence, RL enables agents to learn optimal decision-making strategies through trial-anderror interactions with the environment. Its strength lies in its ability to derive context-aware policies that adapt to real-time variations in system state and workload. When applied to MEC, RL facilitates the development of intelligent resource allocation strategies that can autonomously explore and exploit the available infrastructure. RL agents learn to balance performance metrics such as latency, energy consumption, and throughput based on feedback signals received from the environment. This enables dynamic adaptation to changes in workload distribution, network topology, and user behavior, ultimately contributing to improved efficiency, scalability, and quality of service in MEC systems.

4. PROBLEM DESCRIPTION AND MODELING

This section presents the system architecture and the mathematical formulation of the joint task offloading and resource allocation problem in the considered MEC environment. We first describe the network architecture and communication model, then formulate the optimization problem and introduce the proposed Multi-Agent Reinforcement Learning framework

4.1. Network Architecture

We consider a multi-access edge computing scenario in which a set of users N = {1, 2, 3,…, N} connect to a single macro base station via Long Term Evolution (LTE) wireless communication. The base station supports multiple users through sub-channel division and is concurrently linked to an edge server that provides computational support for offloaded tasks [27], as illustrated in Figure 2.

Each user is equipped with a heterogeneous device such as a smartphone, tablet, or portable computing unit. The edge infrastructure comprises peripheral servers with varying processing capabilities and workload conditions. In this distributed environment, users generate computational tasks and compete forthe limitedresources available at the MEC servers. To maximize performance, users aim to offload their tasks to the most suitable edge nodes, ideally those with higher processing capacity. However, when multiple users simultaneously target highperformance servers, resource contention may occur. This can lead to congestion, degraded performance, and increased latency, negatively impacting the overall quality of experience. Therefore, dynamic and efficient resource allocation mechanisms are necessary to handle varying load patterns and ensure service reliability across the network.

Figure 2. Single-cell multi-user scenario in Multi-Access Edge Computing Offloading

We consider a scenario in which N user devices (UDs), indexed from 1 to N, are connected within a single cellular coverage area through a centralized base station, commonly referred to as an eNodeB (eNB). This base station is directly connected to an Edge Computing Server (ECS), which is responsible for executing offloaded computational tasks. Each user device continuously generates computation-intensive tasks. Depending on the current network conditions and resource availability, a device can either process the task locally using its internal resources or offload it to the edge server via the eNB for remote execution. This decision, whether to compute locally or offload, is critical for optimizing both system performance and user experience in a dynamic MEC environment.

4.2. System Model

4.2.1. Communication Model

We consider a scenario in which multiple users share access to a common base station (eNodeB, or eNB), enabling wireless communication with minimal interference among users.

When N users offload their computational tasks simultaneously, the total wireless bandwidth is equally distributed among them. Accordingly, the downlink rate for the nth user device is given by:

In this expression, W denotes the total available wireless bandwidth, K is the number of active offloading users, Pn represents the transmission power of user n, hn is the channel gain, and N0 is the variance of the complex white Gaussian noise.

4.2.2. Task Model

Each user equipment (UE) is assumed to generate a computationally demanding task, denoted by Rn(Bn, Dn, τn). This task may either be executed locally on the device or offloaded to a nearby MEC server. Here, Bn refers to the input data size (including parameters and code), Dn indicates the number of CPU cycles required to complete the task, and τnrepresents the maximum acceptable delay. A direct correlation is assumed between Bnand Dn, and the value of Dnremains unchanged whether the task is processed locally or remotely. These parameters vary depending on application type and can be estimated through task profiling. We further assume that tasks are indivisible; each task must be either fully executed locally or entirely offloaded. To represent the offloading decision, we define a binary variable αn∈ {0, 1}. The decision vector for all users is denoted by A = [α1, α2, …, αn]. When αn= 0, the task is processed locally by the UE, conversely, αn= 1 indicates offloading to the MEC server.

4.2.3. Local Execution Model

If a user decides to execute its task Rnlocally, the execution time depends on the computational capacity of the device, denoted by f 1 n (in CPU cycles per second). The local execution time for task Rnis is calculated as:

The corresponding energy consumption is given by:

where znis the energy consumed per CPU cycle. Based on empirical measurements [28], we set zn= 10-27 (𝑓ₙˡ) 2 . To capture the trade-off between time and energy, we define the total local execution cost as a weighted sum:

Here, 𝐼𝑛 𝑡 and 𝐼𝑛 𝑒 are the weights assigned to the time and energy components, respectively. These weights satisfy 0 ≤ I ≤ 1 and 𝐼𝑛 𝑡+ 𝐼𝑛 𝑒= 1, and are assumed to be constant for the entire offloading decision process.

4.2.4. Offloading Model

When user equipment (UE) opts to offload its task Rn to the MEC server, the process involves three sequential phases. First, the UE transmits the input data including program instructions and task parameters to the eNB via the wireless access network. The eNB then forwards these data to the MEC server, which allocates a portion of its available computing resources to execute the task on behalf of the user. Finally, the processed results are returned to the UE. The duration of the initial offloading phase corresponds to the transmission time of the input data:

Here, rndenotes the achievable download rate of the UE, as defined earlier in the communication model. The energy consumption associated with this transmission is expressed as:

The second phase concerns the execution of the task at the MEC server. The corresponding processing time isgiven by:

In this context, fnrefers to the computing resources (CPU cycles per second) allocated to user n, and F denotes the total available capacity of the MEC server. To ensure fairness and avoid resource overuse, we impose the constraint ∑ α𝑛𝑓𝑛 𝑃𝑁 𝑛=1 ≤ 𝐹. During the processing phase, the UE remains idle. The energy consumed in this inactive state is modeled as:

The third and final phase involves the transmission of the processed output back to the UE. The required time is calculated as:

Here, Bbdenotes the size of the result data, and rbrepresents the corresponding downlink transmission rate. As noted in [8], this data is typically much smaller than the original input, and the download speed is considerably higher. Therefore, both the delay and energy consumption in this phase are considered negligible in our model. Combining the previous expressions, the total execution time for task Rn using the offloading strategy is:

Similarly, the total energy consumed is given by:

The overall offloading cost is modeled as a weighted combination of the execution time and energy consumption:

The overall offloading cost is modeled as a weighted combination of the execution time and energy consumption:

Finally, the total system cost for all users is expressed as:

Here, αn∈ {0, 1} represents the offloading decision of user n. A value of αn = 0 indicates local execution, whereas αn = 1 implies that the task is offloaded to the MEC server.

4.3. Problem Formulation

The objective of this work is to optimize resource utilization in a Multi-Access Edge Computing (MEC) system by jointly minimizing task execution delay and energy consumption across all users. This optimization problem naturally arises as a multiobjective decision-making task subject to constraints on computational capacity and service latency, which are essential for maintaining Quality of Service (QoS).

Formally, the objective is to minimize the total system cost while ensuring that all tasks are completed within acceptable delay limits and that the processing capacity of the MEC server is not exceeded.

The optimization problem is defined as follows:

The problem is formulated as a multi-objective optimization task that jointly considers latency and energy consumption. It is addressed using a weighted-sum approach, which converts the latency–energy trade-off into a single scalar objective. This allows tractable optimization while maintaining flexibility through adjustable weights that reflect different user preferences.

In this formulation, 𝐴 = [𝛼1, 𝛼2, … , 𝛼𝑁]represents the offloading decision vector, where 𝛼𝑛 = 0indicates local execution and 𝛼𝑛 = 1denotes offloading to the MEC server. The vector 𝑓 = [𝑓1, 𝑓2, … , 𝑓𝑁]defines the computational resources allocated to each user at the MEC server. The constraints are interpreted as follows: (C1) enforces binary offloading decisions for each task; (C2) ensures that task execution delay does not exceed the maximum tolerable threshold 𝜏𝑛; (C3) limits the resources allocated to each individual user based on server capacity; and (C4) guarantees that the total allocated resources across all users do not exceed the overall MEC server capacity 𝐹.

Due to the binary decision variables and the coupling introduced by shared computational resources, the resulting optimization problem is non-convex and combinatorial. Consequently, its complexity increases exponentially with the number of users, making traditional optimization methods computationally intractable for large-scale scenarios.

4.4. Multi-Agent Reinforcement Learning Framework

In this work, the joint task offloading and resource allocation problem in Multi-Access Edge Computing (MEC) is modeled as a decentralized Multi-Agent Reinforcement Learning (MARL) framework. Each user–base station (U2B) link is represented as an autonomous learning agent interacting within a shared MEC environment. Agents make decisions based on local observations, including channel conditions, task characteristics, residual energy, and available computing resources, which reflects the partial observability inherent to practical MEC systems.To address scalability and non-stationarity issues arising from simultaneous learning, we adopt the Centralized Training and Decentralized Execution (CTDE) paradigm. During the training phase, agents share experience information to learn a centralized value function or Q- network, while decisions are executed independently at runtime using only local observations. This paradigm enables coordinated learning while preserving decentralized operation, and has proven effective for cooperative multi-agent systems [29].

The MEC environment is formulated as a Markov Decision Process (MDP), characterized by the following components:

• State (s): The global state captures system-level information such as the aggregated execution cost and the remaining computing capacity of the MEC server. Each agent n observes a local partial state sn, derived from the global state.

 • Action (an): At each decision step, agent n selects an action consisting of a task offloading decision and a corresponding computing resource allocation.

 • Reward (rn): The reward is defined as a negative function of the execution cost, which includes both latency and energy consumption. Penalty terms are incorporated to discourage violations of service-level agreements (SLAs) or excessive resource usage.

 The objective of each agent is to learn a policy πn(an|sn) that minimizes its long-term expected cost while contributing to overall system efficiency. Although agents operate independently, they are cooperative in nature, as they share a common global objective through the reward formulation. This implicit coordination encourages balanced resource utilization and avoids selfish behavior.

 To improve training stability and convergence in the presence of non-stationarity, experience replay buffers and target network updates are employed. These mechanisms reduce temporal correlation between samples and smooth policy updates, allowing agents to adapt effectively to dynamic MEC conditions. The proposed cooperative MARL framework is therefore well – suited to decentralized and resource-constrained edge computing environments.

5. MULTI-AGENT REINFORCEMENT LEARNING FOR RESOURCE ALLOCATION IN MEC

In this section, we formulate the joint task offloading and resource allocation problem as a MultiAgent Reinforcement Learning (MARL) process. The MEC system is modeled as a Markov Decision Process (MDP), where agents interact with the environment and learn adaptive policies through trial-and-error in order to minimize the long-term execution cost.

State space: The system state is defined by the tuple s = (tc, ac), following the formulation in [30,31]. The variable tcrepresents the total cost of the system, defined as tc= Call, which aggregates the execution cost of all users. The variable ac denotes the remaining computational capacity of the MEC server and is expressed as:

where F is the total computing capacity of the MEC server and fnis the computing resource allocated to user n. Each agent observes a local and partial view of the global state, reflecting realistic MEC environments with limited observability.

Action space: At each decision step, the action taken by the agents consists of two components: (i) the task offloading decision and (ii) the allocation of computing resources at the MEC server. The offloading decision is represented by the binary vector:

A = [α1, α2, . . . , αN],where αn∈ {0, 1} indicates whether user n executes its task locally or offloads it to the MEC server. The resource allocation decision is represented by the vector

f = [f1, f2, . . . ,fN],

where fndenotes the amount of computing resources allocated to user n. The overall action vector is therefore defined as:

a = [α1, α2, . . . ,αN , f1, f2, . . . ,fN].

Reward function: After executing an action ain state s, agents receive a reward R(s, a) that reflects the quality of the decision. Since the objective of the optimization problem is to minimize the total cost of the system, the reward is defined as a negative function of the cost. This design ensures a direct correspondence between maximizing cumulative reward and minimizing execution latency and energy consumption. Additional penalty terms are applied when system constraints, such as delay deadlines or MEC server capacity limits, are violated.

Agents: Each user–base station (U2B) link is modeled as an autonomous learning agent operating under constrained processing and energy resources. Agents make decisions based solely on local observations while interacting within a shared MEC environment. Although agents act independently, they pursue a common global objective through a shared reward structure, which promotes cooperative behavior. This MARL formulation is well suited to decentralized MEC systems, where scalability and adaptability are essential.

5.1. Q-Learning algorithm

Q-learning is a reinforcement learning algorithm that enables an agent to learn to make decisions in an environment to maximize long-term rewards. The agent maintains a table called the Qtable, which contains the expected reward values for each pair (state, action). The algorithm uses interactions with the environment to iteratively update these values. The agent chooses an action based on the current state of the environment and the Q value associated with each action. Then it executes this action, observes the new state and the associated reward, and updates the corresponding Q value using a learning formula.

The algorithm repeats these steps for many iterations until the Q values converge to accurate estimates of long-term rewards. The Bellman optimality equation is given by:

Q represents the Q-table that stores the Q-values for state-action pairs, M is the total number of episodes, α is the learning rate, γ is the discount factor.

5.2. Deep Q-learning algorithm

Deep Q-Learning (DQL) is an advanced reinforcement learning approach that combines Qlearning with deep neural networks to handle complex decision-making problems. In DQL, a neural network estimates Q-values for different actions based on the current state. The model is trained by minimizing the difference between predicted and target Q-values using replay memory, which stores past experiences to improve learning stability and reduce data correlation. Additionally, DQL applies an exploration–exploitation strategy, such as the ϵ-greedy method, to balance random exploration and optimal action selection for maximizing long-term rewards.

θ represents the parameters of the neural network, N is the capacity of the replay memory, M is the total number of episodes, ϵ is the exploration rate, γ is the discount factor, α is the learning rate

5.3. Double Deep Q-learning algorithm

Double Deep Q-Learning (DDQL) is an extension of Deep Q-Learning that improves learning stability by using two neural networks instead of one. The first network estimates current Qvalues, while the second network computes target Q-values for training. This separation between action selection and value estimation helps reduce overestimation bias and enhances the accuracy and stability of the learning process.

Figure 3. Architecture of Double Deep Q-Learning Algorithm

In the algorithm mentioned above, θ denotes the parameters of the neural network, N is the memory capacity of the replay, M is the total number of episodes, ϵ is the exploration rate, γ is the discount factor, α is the learning rate, and C is the target network update frequency.

5.4. Training Setup and Stability Mechanisms

The reinforcement learning algorithms (Q-learning, DQN, and DDQN) were trained through episodic interactions within a simulated MEC environment. In each episode, the environment generates dynamic task arrival scenarios with randomly initialized users, workloads, and communication conditions. During training, agents interact with the environment over several timesteps, receive rewards, and iteratively improve their decision-making policies.

To enhance convergence and stabilize learning, especially in the multi-agent setting, several mechanisms were adopted. First, an experience replay buffer of size 10,000 was used to store transitions (𝑠 , 𝑎 , 𝑟 , 𝑠 ′ ), from which random mini-batches of 64 samples were selected to reduce temporal correlations and improve sample efficiency. Second, DQN and DDQN employed separate target networks to stabilize Q-value estimation, with updates performed every 𝐶 = 10episodes. Finally, DDQN applied a soft update strategy to gradually synchronize the target network with the main network, further improving training stability.

θ ′ ← τθ + (1 − τ)θ ′ , τ = 0.01

To further improve learning stability, DDQN applies a soft update mechanism that gradually aligns the target network with the main network, making the training process more robust in nonstationary environments. In addition, agents adopt an ε-greedy exploration–exploitation strategy, where the exploration rate decreases exponentially from 0.9 to 0.01 to favor exploration during early training stages and exploitation afterward.

The algorithms were trained for 500 episodes, and convergence was evaluated by monitoring the average reward and cost per episode. Early stopping was introduced once the learned policy became stable. To address the challenges of multi-agent non-stationarity, a centralized training and decentralized execution (CTDE) paradigm was employed, where agents share a common Qfunction during training while making decisions independently during deployment. Overall, this training framework improves the stability, convergence, and adaptability of the learned policies in dynamic MEC environments.

6. SIMULATION RESULTS

In this section, we evaluate the performance of the proposed MARL framework through extensive simulations under different MEC configurations. The experiments analyze convergence behavior, total system cost, scalability, and the impact of varying task and network parameters.

6.1. Scenario and Parameters

To address concerns about the robustness of our approach, we present here the critical parameters used in the Q-Learning, DQL and DDQL algorithms, and their impact on system performance. The table below summarizes these parameters and their typical values in the literature. These parameters were used as the basis for a sensitivity analysis, in which we individually varied each factor within a range around its typical value to observe the impact on convergence, stability, and quality of the solutions obtained. The set of hyperparameters used to train each reinforcement learning algorithm is summarized in Table 2.

Table 2. Reinforcement Learning Hyperparameters Used in Training

While Q-learning uses a simple tabular update, DQL and DDQL leverage experience replay and target networks with different update strategies. Note: The target network update strategies differ across the deep RL methods. DQL applies a hard update every fixed number of episodes (e.g., 10), while DDQL employs a soft update where the target network θ′ is updatedincrementally via:

𝜃 ′ ← 𝜏𝜃 + (1 − 𝜏)𝜃 ′ , 𝜏 = 0.01

This soft update stabilizes learning under non-stationary multi-agent environments. Replay buffers are used only in deep RL algorithms to break correlation between sequential observations. The convergence behavior of the three RL models is illustrated in Figure 4.

Figure 4. Comparison of training convergence for Q-Learning, DQN, and DDQN over 500 episodes.

As shown, DDQN achieves faster and more stable convergence, while DQN exhibits slower learning with moderate fluctuation. Q-Learning shows the weakest convergence trend due to its limited generalization capability in high-dimensional state spaces. Having demonstrated the convergence and stability of the training phase, we now evaluate the effectiveness of each reinforcement learning approach under various system configurations. The following experiments compare Q-Learning, DQN, and DDQN with respect to key performance indicators including total system cost, energy consumption, and execution latency. These metrics are analyzed under different numbers of users, varying task loads, and MEC server capacities.

The simulation is carried out under the following scenario, where we consider a configuration involving a single cell with a bandwidth of W = 10 MHz. An eNB is deployed with an MEC server located in the center. End-users (UEs) are randomly distributed within a radius of 200 meters around the eNB. The computing capacity of the MEC server is F = 6 GHz, with each UE having a CPU frequency of 𝑓ₙˡ= 1 GHz.

The parameters for UE transmit power and idle power are configured as Pn=500 mW and 𝑃𝑛 𝑖 = 100𝑚W, respectively. We assume that the data size of the computational load, denoted as Bn(in kbits), follows a uniform distribution between 300 and 500, while the number of CPU cycles required, denoted as Dn, follows a uniform distribution between 100 and 1000. For simplicity, the decision weight for each end user is set to 𝐼𝑛 𝑡 + 𝐼𝑛 𝑒 = 0.5.. We evaluated the proposed algorithms by comparing them with two other methods, based on the following parameters. The” Full Local” option implies that all users perform their tasks using their own local computing resources. The” Full Offload” option means that all users delegate their tasks to the server MEC and that all F computing resources are distributed equally among all users. We begin by presenting the total cost of the MEC system as a function of the increase in the total number of end users in the network, with the computing capacity of the MEC server set at F = 6 GHz.

6.2. Discussions

This section presents the simulation results, which allow an evaluation of the performance of the proposed methods. The results show a consistent trend where the total cost associated with all five methods increases as the number of UEs grows.

In Figure 5, the proposed MARL approach consistently outperforms the others, followed by DQN and Q-learning. Although the Full Offload strategy shows a slightly lower cost than DQN and Q learning at the 3 UE mark, its cost increases more sharply as the number of UEs increases.

In general, MARL indicates superior performance in all scenarios. As the number of users increases, the MEC server’s capacity becomes insufficient to accommodate all offloading demands. This limitation underscores the importance of carefully selecting which users should offload their tasks, as attempting to serve too many users can degrade overall performance. Consequently, the user selection strategy plays a critical role in ensuring efficient offloading under constrained resources.

Figure 5 . Total cost versus number of UEs

Figure 6 illustrates how the total cost of the MEC system evolves as a function of two variables: the computing capacity of the MEC server and the data size (Bn) of the download task, with a fixed number of UEs at 10. Increasing the size of offloaded data leads to higher total costs in all methods, driven by higher energy consumption during the offloading process. This increase in resource usage directly contributes to the overall cost of the MEC system. Among the methods, the proposed MARL approach consistently achieves better performance, mainly because of its slower cost growth compared to the others. In contrast, the Full Local method shows a much steeper increase as the data size grows, highlighting its inefficiency for larger tasks. This suggests that as the taskdata size increases, the Full Local method struggles more with delay and energy usage. These observations suggest that larger data sizes significantly impact the cost of the MEC system, mainly due to the positive correlation between Dn and Bn, which jointly drives up the objective function.

Figure 7 illustrates that the proposed MARL method delivers the best performance, while DQN and Q-learning show only a slight deviation from the DDQN method. Notably, the Full Local curve remains constant regardless of the MEC server capacity, as local computation does not rely on MEC resources. In contrast, the other methods show a decrease in total cost with increasing MEC server capacity, due to the corresponding reduction in execution time. These results confirm that DDQN provides a better stability–performance trade-off in dynamic MEC environments.

Compared with existing reinforcement learning-based MEC approaches reported in the literature [13–19], the proposed DDQN-based MARL framework demonstrates improved convergence stability and lower overall system cost under dynamic resource conditions. Unlike several centralized MEC optimization methods, the proposed framework operates under decentralized execution with partial local observations, making it more suitable for practical large-scale MEC deployments. In addition, the CTDE-based coordination strategy improves scalability while reducing the communication overhead between agents and the MEC server.

6.3. Feasibility and Deployment Considerations

Although the proposed MARL framework indicates strong performance in simulation, real-world deployment in MEC systems introduces several challenges, including computational overhead, communication constraints, and training requirements. Deep reinforcement learning models such as DQN and DDQN require higher computational resources compared to tabular Q-learning. In distributed environments, centralized training may introduce latency due to model synchronization. To mitigate these limitations, offline training followed by periodic policy updates represents a practical deployment strategy. Overall, the proposed approach remains feasible under resource-aware MEC architectures

6.4. Computational Complexity Analysis

Compared to Q-Learning, which has constant update complexity O(1), DQN and DDQN introduce higher computational cost due to neural network forward and backward propagation, with complexity approximately O(n·d), where n is batch size and d is number of network parameters. Although more computationally expensive, deep RL methods provide better scalability and generalization in high-dimensional MEC environments.

6.5. Experimental Robustness and Statistical Validity

To ensure robustness, each simulation scenario was repeated 10 times with different random seeds. Performance metrics are averaged over these runs. The low variation across experiments confirms the stability and reliability of the proposed approaches. These results demonstrate statistical consistency under dynamic system conditions.

7. CONCLUSION

This study has demonstrated the effectiveness of reinforcement learning algorithms, namely QLearning, DQN, and DDQN, for optimizing resource allocation in multiuser Multi-Access Edge Computing (MEC) systems. Through extensive simulations, we compared these approaches with traditional strategies such as Full Local execution and Full Offloading under varying system conditions. Among the tested algorithms, DDQN consistently achieved the best performance in terms of minimizing overall system cost. Its ability to dynamically balance task offloading between local devices and the MEC server resulted in significant gains in efficiency and energy savings. While DQN and Q-Learning also showed competitive results, the superior convergence and stability of DDQN confirm its suitability for dynamic and distributed environments. Beyond quantitative results, we also addressed key practical considerations, including training stability, deployment feasibility, and computational complexity. These insights reinforce the applicability of MARL-based approaches in real world MEC scenarios.Although the proposed framework achieved promising results, the current study is limited to simulation-based evaluation under a single-cell MEC scenario. Real-world deployment may introduce additional constraints related to user mobility communication failures, and distributed synchronization between agents.

Future work will explore more advanced settings, such as federated or privacy-aware MARL, and evaluate the proposed models using real world workloads and deployment testbeds in future wireless edge networks. Such directions will help further validate the robustness, scalability, and generalizability of the proposed framework.

CONFLICTS OF INTEREST

The authors declare no conflict of interest

REFERENCES

[1] Akpakwu, G.A., Silva, B.J., Hancke, G.P., Abu-Mahfouz, A.M.: A Survey on 5G Networks for the Internet of Things: Communication Technologies and Challenges. IEEE Access 6, 3619–3647 (2018) https://doi.org/10.1109/ACCESS.2017.

[2] Vieira, J.L., Macedo, E.L.C., Battisti, A.L.E., Noce, J., Pires, P.F., MuchaluatSaade, D.C., Oliveira, A.C.B., Delicato, F.C.: Mobility-aware SFC migration in dynamic 5G-Edge networks. Computer Networks 250, 110571 (2024) https://doi. org/10.1016/j.comnet.2024.110571.

 [3] Pham, Q.-V., Fang, F., Ha, V.N., Piran, M.J., Le, M., Le, L.B., Hwang, W.-J., Ding, Z.: A Survey of Multi-Access Edge Computing in 5G and Beyond: Fundamentals, Technology Integration, and Stateof-the-Art.IEEEAccess8,116974–117017(2020) https://doi.org/10.1109/ACCESS.2020.3001277.

[4] Macedo, E.L.C., Battisti, A.L.E., Vieira, J.L., Noce, J., Pires, P.F., MuchaluatSaade, D.C., Oliveira, A.C.B., Delicato, F.C.: Distributed Auction-Based SFCPlacement in a Multi-domain 5G Environment. SN Computer Science 5(1), 48(2023) https://doi.org/10.1007/s42979-023-02291-1

 [5] Shyalika, C., Silva, T., Karunananda, A.: Reinforcement learning in dynamic taskscheduling: A review. SN Computer Science 1(6), 306 (2020)

 [6] Khan, A.u.R., Othman, M., Madani, S.A., Khan, S.U.: A Survey of MobileCloud Computing Application Models. IEEE Communications Surveys & Tutorials 16(1), 393–413 (2014) https://doi.org/10.1109/SURV.2013.062613.00160.

[7] Yu, Y.: Mobile edge computing towards 5G: Vision, recent progress, and open challenges. China Communications 13(Supplement2), 89–99 (2016) https: //doi.org/10.1109/CC.2016.7833463.

[8] Zhang, K., Mao, Y., Leng, S., Zhao, Q., Li, L., Peng, X., Pan, L., Maharjan,S., Zhang, Y.: EnergyEfficient Offloading for Mobile Edge Computing in 5G Heterogeneous Networks. IEEE Access 4, 5896–5907 (2016) https://doi.org/10. 1109/ACCESS.2016.2597169.

[9] Nunna, S., Kousaridas, A., Ibrahim, M., Dillinger, M., Thuemmler, C., Feussner, H., Schneider, A.: Enabling Real-Time Context-Aware Collaboration through 5G and Mobile Edge Computing. In: 2015 12th International Conference on Information Technology – New Generations, pp. 601–605 (2015). https://doi.org/ 10.1109/ITNG.2015.155.

 [10] Stojmenovic, I., Wen, S.: The fog computing paradigm: Scenarios and securityissues. Federated Conference on Computer Science and Information Systems, 1–8 (2014)

[11] Kyung, Y.: Prioritized task distribution considering opportunistic fog computingnodes. Sensors 21(8), 2635 (2021)

 [12] Verma, V.R., Pushkar, kumar, B., Verma, A., Sharma, V., Tripathi, P.K.: Anextensive investigation on lyapunov optimization-based task offloading techniques in multi-access edge computing. SN Computer Science 6(6), 603 (2025)

[13] Li, H., Ota, K., Dong, M.: Deep reinforcement scheduling for mobile crowdsensingin fog computing. ACM Transactions on Internet Technology 19(2), 1–18 (2019)

[14] Kumar, R., Singh, P.: Deep Reinforcement Learning-Based Task Offloading in Edge Computing. International Journal of Computer Networks & Communications (IJCNC), 15(2), 2023.

[15] Priyadarshni, Kumar, P., Kadavala, D., Tripathi, S., Misra, R.: An integratedapproach of efficient edge task offloading using deep rl, attention and mdstechniques. SN Computer Science 5(6), 681 (2024)

 [16] Zhang, J., et al.: Value decomposition networks for multi-agent reinforcementlearning in mobile edge computing. IEEE Internet of Things Journal (2021). [17] Xu, B.e.a.: Actor-critic based multi-agent reinforcement learning for task offloading in mec. IEEE Access (2020)

 [18] Wang, Y.e.a.: Federated reinforcement learning for distributed mec systems. IEEETransactions on Network and Service Management (2023).

[19] Elhaou, H.e.a.: Machine learning for user mobility management in a mobile fogcomputing environment. Cluster Computing (2025).

 [20] Zhang, Y.: Mobile Edge Computing. In: Zhang, Y. (ed.) Mobile Edge Computing. Simula SpringerBriefs on Computing, pp. 9–21. Springer, Cham (2022). https: //doi.org/10.1007/978-3-030- 83944-4. https://doi.org/10.1007/978-3-030-83944-4.

 [21] Osibo, B.K., Jin, Z., Ma, T., Marah, B.D., Zhang, C., Jin, Y.: An edgecomputational offloading architecture for ultra-low latency in smart mobile devices. Wireless Networks 28(5), 2061–2075 (2022) https://doi.org/10.1007/ s11276-022-02956-4.

 [22] Liu, H., Eldarrat, F., Alqahtani, H., Reznik, A., Foy, X., Zhang, Y.: MobileEdge Cloud System: Architectures, Challenges, and Approaches. IEEE Systems Journal 12(3), 2495–2508 (2018) https://doi.org/10.1109/JSYST.2017.2654119.

 [23] Lei, X., Li, Q., Bo, P., Zhou, Y.Z., Chen, C., Peng, S.L.: Long Short-Term Deterministic Policy Gradient for Joint Optimization of Computational Offloading and Resource Allocation in MEC. In: Tari, Z., Li, K., Wu, H. (eds.) Algorithms and Architectures for Parallel Processing, pp. 329–348. Springer, Singapore (2024).https://doi.org/10.1007/978-981-97-0811-6 20

 [24] Sun, X., Li, W., Liu, H., Fang, J., Wen, Z., Wen, C.: A task offloading strategybased on sequential waiting model in MEC. Multimedia Tools and Applications 83(18), 54473–54493 (2024) https://doi.org/10.1007/s11042-023-17578-x.

 [25] Sun, X., Li, W., Liu, H., Fang, J., Wen, Z., Wen, C.: A task offloading strategy based on sequential waiting model in MEC.Multimedia Tools and Applications 83(18), 54473–54493 (2024).

 [26] Mahbub, M., Shubair, R.M.: Contemporary advances in multi-access edge computing: A survey of fundamentals, architecture, technologies, deployment cases,26security, challenges, and directions. Journal of Network and Computer Applications 219, 103726 (2023) https://doi.org/10.1016/j.jnca.2023.103726.

 [27] Zhang, Z., Li, C., Peng, S., Pei, X.: A new task offloading algorithm inedge computing. EURASIP Journal on Wireless Communications and Networking 2021(1), 17 (2021) https://doi.org/10.1186/s13638-021-01895-6.

[28] Khoda, M.E., Razzaque, M.A., Almogren, A., Hassan, M.M., Alamri, A., Alelaiwi,A.: Efficient Computation Offloading Decision in Mobile Cloud Computing over 5G Network. Mobile Networks and Applications 21(5), 777–792 (2016) https: //doi.org/10.1007/s11036-016-0688-6.

 [29] Zhang, J.e.a.: Value decomposition networks for marl in mobile edge computing.IEEE Internet of Things Journal (2021).

 [30] Gao, H., Guo, X.: Deep Reinforcement Learning-Based Computation Offloadingand Optimal Resource Allocation in Industrial Internet of Things with NOMA. In: 2022 11th International Conference on Communications, Circuits and Systems (ICCCAS), pp. 198–203 (2022). https://doi.org/10.1109/ICCCAS55266.2022. [31] Jadad, H., &Touzene, A. (2026). QOS-BASED COST-EFFECTIVE OFFLOADING AS A SERVICE MIDDLEWARE FOR MOBILE CLOUD APPLICATIONS. International Journal of Computer Networks and Communications, 18(2), 1-17.

 [32] Bhatt, R., Mehra, R., & Upreti, K. AN ADAPTIVE HYBRID SCHEDULING APPROACH FOR SUSTAINABLE AND RELIABLE CLOUD SERVICES. [33] Pham, H.A., Nguyen, D., Nguyen, T.X.U., Nguyen, T.C.N., Ngo, M.N.: Deep Reinforcement Learning-Based Resource Allocation in Massive MIMO NOMA Systems. International Journal of Computer Networks & Communications (IJCNC), 17(6), 2025.

[34] Gad-Elrab, A.A.A., Alsharkawy, A.S., Embabi, M.E., Sobhi, A., Emara, F.A.: Adaptive MultiCriteria-Based Load Balancing Technique for Resource Allocation in Fog-Cloud Environments. International Journal of Computer Networks & Communications (IJCNC), 16(1), 2024.

 [35] Le, H.B., Vo, T.T., Le, D.H.: IRLR: an Improved Reinforcement Learning-Based Routing Algorithm for Wireless Mesh Networks. International Journal of Computer Networks & Communications (IJCNC), 16(4), 2024

AUTHORS

Youssef OUKISSOU is a doctor’s degree in Business Intelligence from the Faculty of sciences and Techniques of Beni Mellal, belonging to the University Sultan Moulay Slimane. He is currently Ph.D. Doctors in IoT and cloud computing resources management at the Faculty of Sciences and Techniques of Beni Mellal. His recent interests include resources management, edge/fog computing, routing, authentication protocols, sensor networks, and IoT.