The Modified Parallelized File Transfer Protocol for Multi-users
Wei-Chen Lin Jiun-Jian Liaw Chiung-Ta Wu
Department of Information and Communication Engineering Chaoyang University of Technology Taiwan(R.O.C.), Taichung s9930615@cyut.edu.tw, jjliaw@cyut.edu.tw
Abstract—File Transfer Protocol is the most popular file transfer standard. This FTP can let a computer system connect to the internet via a communication network, and it can also access the system resources from the internet. File server is being downloaded now by TCP/IP protocol, while before it was downloaded by signal server. Due to the limited flow control, the old method could not use download time efficiently. In order to solve the flow control problem, some researchers came up with a distributed file transfer method that can share the document with other users, thus allowing the shortest transfer time using neighboring servers. Although the distributed file transfer can reduce the file transfer time, it was not concerned with the quality of the transfer between two servers, and was inappropriate for file segmentation and transfer, so the file transfer could not attain the optimal transfer efficacy. Based on our research, we propose a new method called Distributed Parallelized File Transfer Protocol (DPFTP). This method is based on Parallelized File Transfer Protocol (P-FTP) and is beneficial to multi-users, as it utilizes server loading and reduces the download time. Keywords—FTP; P-FTP; DPFTP ; multi-users;

(Autonomous System) P-FTP server in the center of the area and message passing to allow the client to download from multiple file servers by parallel transfer. The P-FTP file transfer protocol, divided into three parts performance, includes the client, PFTP server and mirror servers; the P-FTP flow chat as shown in fig 1[2]. II. PARALLELIZED FILE TRANSFER PROTOCOL

I.

INTRODUCTION

Due to the development of Internet, the data transfer by TCP/IP protocol [1] has been increasing; as a result, the signal server cannot satisfy users' demand anymore. The aim is to reduce the download time of the shared document distributed through the internet by using neighboring topology. Although the previous researchers propose to restore to Client-Server architecture by using P-FTP (Parallelized File Transfer Protocol) [2], it can cause the P-FTP server overload when the number of users increases. In a previous research, Guyton and Schwartz have come up with a theory about reaching the highest transfer efficiency by Hop Counts and Round trip. In addition to the above, Carter and Crovella have put forward the idea [3] to choose user and mirror servers by using bandwidth and congested paths. Venkatasubramanian has suggested to choose the file server by CPU cycle, I/O bandwidth, and memory usage amount [4]. All these ways use files from the server's network environment to split files and achieve appropriate allocation. This distributes the download work to each file server and increases the transfer efficiency. For large file transfers, researchers Sohail, Jha and Elgindy…etc. have proposed a method called P-FTP (Parallelized File Transfer Protocol), which uses AS

Figure 1. Interaction among P-FTP entities The process description of P-FTP as below: 1) When the user needs to download a file, the client sends a Request Message (RM) to the P-FTP server. 2) After the P-FTP server receives a request message, it begins to collect all the messages with server utilization data from the mirror servers. Then it calculates the number of selected mirror servers the message has to pass through, and the client delay time. According to these parameters for proper applicability algorithm, the server calculates and selects the mirror server with the best transfer efficiency, and the file size in accordance with the transmission efficiency of the server segments, then it assigns the file to each of the mirror servers. Finally, the P-FTP server sends an Inform Message (IM) to four mirror servers. The inform message includes the requirements of the sub-file sizes and requests the file location and the client's location.

718

3) A server receives an inform message and saves it, then replies with a Confirm Message (CM) to the P-FTP server. 4) When P-FTP server receives the confirmation message, the server will send a Reply Message (RM) to the client. This reply message contains the mirror server parameters and location information. 5) Usually the client transfers the file to all mirror servers, and after the transfer is completed, each mirror server sends a completed message to the P-FTP server. 6) When the P-FTP server detects environment changes in the network, it re-evaluates the appropriateness of all operations of the mirror servers and after adjusting to the mirror servers sends the message again. 7) If the P-FTP server detects that all mirror servers fail or are unavailable, it sends “Cannot Use” message to the client and aborts the file transfer. 1. In a previous study, researchers have proposed a method of P-FTP file transfer without a central server (Modified File Transfer Protocol, MFTP), mainly to improve the P-FTP server, because a central operator requires a large amount of computation and consumes time. If the number of users increases, it affects the P-FTP server, adding a heavy load. The MFTP [5] approach allows clients to find the nearest file server and use it as the computing server, and it lets the server's computing support online to help find the nearby file servers, whose load can be distributed across the network by using parallel file transfer. III. MODIFIED FILE TRANSFER PROTOCOL

for each file server then returns a message to the client. This response message contains the supported file server location, as well as the file distribution ratio. Appropriate algorithms can be divided into Best Utilization Policy (BUP) [6] and the Lowest Delay Policy (LDP) [6], calculated by the following equations:

BUP defined as

Sk =

Vk m k =1

Vk

(1)

LDP defined as

sk =

Vz Vk

(2)

Sk =

sk m sk k =1

(3)

In the above calculation,

represents the number of mirror

servers, k represents mirror server's time delay information, z defines the minimum delay time of a mirror server, k is based on the ratio of k and z It is used to select the mirror server and obtain the optimal sub-file segmentation. 6) Each response message contains the downloaded file information. After a client receives the response message for parallel file downloads about the beginning of each support file to FTP server, he can use this message to defragment files. 7) When the network environment changes, the client will re-sent request messages to the computing server by periodically requested information includes the current portion of the file being transferred that is not yet completed, while the other servers inform the computing server about the remaining file transfer section that has not yet been completed. Then the computing server will recalculate appropriateness, so that all file servers can dynamically adjust the speed of the file transfer. 8) Each file sends a complete message telling the client that the file transfer has been finished. After the client receives the complete message, he can defragment the files. In the file transfer process, the file transfers are obtained by the client from the nearest server, which finds support files and does not require any central server. So when the number of users increases, the server does not need to compute something like P-FTP file servers or deal with multiple users. Thereby the computational burden is reduced, so MFTP achieves load balancing easier than P-FTP does. In previous studies, the MFTP file transfer process can indeed reduce the P-FTP central server computational burden. The MFTP approach is to use the mirror servers nearest to the client as file servers. Due to the proximity to the server location, when MFTP finds the support server, the transmission efficiency is higher than that of P-FTP central server, and the transfer time is also reduced. Although the above two methods (P-FTP and MFTP) both

While P-FTP cannot choose the best server for the client, the improved MFTP approach allows the client to choose a central server. The process of MFTP is as below: 1) When a client wants to download a file, he will send a multicast message to inform all the mirror servers in the area, locate the first responding mirror server and select it as the main server. Then the client sends a request message for the main server to calculate. 2) When main server receives a request message, it will attempt to establish a connection and file transfer via FTP to the client. 3) In addition to sending data, the main server also sends server support multicast messages to all the mirror servers in the AS region. These multicast support messages request clients' location and other information. 4) Mirror servers in the AS area receive support messages and reply to main server immediately. This reply message only contains the server's location information. 5) When the main server receives reply messages, it only selects three mirror servers which have returned support message the fastest, as well as the main server from one of the file servers. First, it uses the servers' reply information to perform the appropriate algorithm to calculate the appropriateness of each file server and allocation of sub-files

719

reduce the transmission time for the main purpose, both of them have a common drawback, which is the need to find a central server as computing server. The central server evaluation as the best server is not necessarily correct, therefore we propose a new method based on P-PFT and designed to ignore the central computing servers, as well as let the client choose the appropriate new file server. This transfer protocol is called DPFTP (Distributed Parallelized File Transfer Protocol). This method allows the client to directly find the best server as shown in figure 2, and also to find mirror servers to calculate and allocate sub-files. This method can conform to the actual transmission delay rate between client and server in order to achieve more rapid results than the P-FTP transmission does. IV. DISTRIBUTED PARALLELIZED FILE TRANSFER PROTOCOL

2) Request reply message, RRM: When an AS region mirror server receives the RM message, it will determine whether a client's requested file corresponds to the RM files according to the relevant parameters. If the file exists, the server must immediately respond with an RRM (Request Reply Message) packet to the client, but if the file does not exist, then this RM packet is discarded. We have designed a simple response packet called RRM packet, which does not contain any information but is mainly used to provide the client with the response time. The client receives the feedback through the RRM packet, and then the RM packet calculates the delay time between servers and the client in order to let the client choose the order of the mirror servers. 3) File Assignment Algorithm, FAA: When a client receives a mirror server's RRM packet in response, it will subtract the time of RRM from the RM time to find the delay time of the mirror server. If the client received the packet earlier, it means the transmission efficiency of the mirror server is higher, and if the client has received a number of RRM packets and calculates the delay time, the appropriate algorithms are implemented. FAA allocates sub-file ratio to each mirror server through the delay time [9] between the client and the server. FFA uses the RRM packet from the first response of a mirror server, selects the minimum delay time of several mirror servers, and in accordance with the delay time splits the file into nonoverlapping sub-files. The mirror server with the smallest delay time will be responsible for a larger number of sub-files, as mirror servers are responsible for the transmission rate in accordance with different transmission ratios of the size of their sub-files. We assume that the network environment does not change, and all mirror servers will complete the transfer at the same time. Sub-file allocation ratio can be calculated by the following formula:

We propose a Distributed Parallelized File Transfer Protocol (DP-FTP) to create a distributed and parallelized environment of a fixed number of file servers. DP-FTP makes users download large size file quickly, and it also balances the loading of file servers when a client requests a file [7]. In our research, the flow chart of the proposed method DPFTP file transfer is shown below:

D1 D FAAi = m i D1 i =1 Di
Figure 2. Interaction among DP-FTP entities 1) Request Message, RM: When a client makes a new request, it will send the request message packet to all mirror servers in AS region. RM packet is mainly used to measure the server status between the client and various links (for example, time delay). When the client side issues the packet, the RM packet will record the time needed to receive the response from mirror servers, so the client can calculate the delay time between the servers. The RM packet consists mainly of information such as: a) Client requests file information (file size, file type, file name). b) If there are some mirror servers in the AS region, the client sends multicast request messages to all mirror servers in the AS region and demands an immediate response [8].

(4)

The FAAi is responsible for a mirror server, responsible for the proportion of the sub-file size,

is is the is

delay time of the mirror server which was the first choice,

selected as the delay time of a mirror server Using the proportion of the sums, it calculates the sub-file size ratio of all servers to the response of the mirror server. When the first mirror server responds with a RRM packet, then the client will calculate the delay time between the first server and receives a certain number of RRM packets (here we set the default number as 4) and begins to perform FAA calculations. The mirror server with the earliest response RRM packet will have the delay time lower than that of all other servers. A higher proportion of sub-files will be assigned to the mirror server with a higher transmission efficiency, and the less efficient transfer mirror server is assigned a fewer sub-file

720

ratio. In the implementation of FAA, we only need to select several mirror servers with the fastest transfer rate for distributed transmission, thus effectively speeding up the file transfer time. 4) (Download Message, DM): The FAA evaluates all the servers responsible for each sub-file size, and the requested file information will be send to each mirror server in DM packets. After a server receives the DM packet, it transmits the file according to the DM packet. The DM packet contains: a. Sub-file related information (file size, from which part of the original file to start the transfer, what percentage of the original file it has). The transfer between the client and mirror servers use file transfer protocol called TF (Transferring File). If the network environment proceeds without any change or packet loss, then all mirror servers will follow the DM information and continue to send the TF files until the transfer is complete.

6) CoM (Complete Message): In order to complete the file transfer, the client will be told to complete the process. When the client receives all CoM messages, he will begin to reorganize files and restore the original files. V. EXPERIMENT In this section, we discuss a simulation performed to compare the performance of P-FTP and MFTP. The experiment is simulated using a common network simulator, Network Simulator 2 (NS2)[10]. The details of the simulation are described below: The simulation experiment was repeated 100 times, and each time the simulation experiments re-generated a new set of links in a network environment with the client. The routing topology was randomly linked to the client router nodes, with a randomly assigned bandwidth and the lowest delay time [11]. Compared with P-FTP, using our method protocol we downloaded 1GB of files. The average aggregated data was enough to achieve the necessary file transfer. In our research, we mainly used P-FTP method to improve our goals. In the process of removing the central computing server, we experimented in order to compare the previous PFTP, MFTP methods with our proposed method DP-FTP, so each experiment used the same topology environment, and simulation was the same for each of the three protocols. In a simulated environment, we focused on the different transmission environments to the simulated experiment and divided it into the following situations. These simulations were designed to verify the download transfer performance. a. b. c. Large file download simulation. Multiple users download at the same time. Multiple users download in interval time. VI. A. EXPERIMENT RESULT

b.

5) Periodic Detection: As the network environment and the transmission speed vary, the FAA calculated distribution ratio cannot be applied to the whole file till the end of the transfer (especially for large file transfers). In the file transfer process, unforeseen conditions may occur (such as a server's crash, network disconnection or power failure, etc.), so in the file transfer process we must periodically test all the mirror servers for their transmission rates. Using a cyclical detection procedure, the client must resend the RM packet to all mirror servers in the AS region and request all mirror servers to respond with RRM packets after receiving their RM packets. After getting the latest transfer rate, once again the FAA is used to calculate the new distribution ratio. This FAA's calculation is performed only for the remaining portion of the file transfer. Periodic detection may encounter the following three problems: a. When the FAA determines that the client selected the same mirror server, but the ratio of division has changed, it will resend a new RM packet to all servers, and the DM packet will contain the new subfile size and location. Then file will be transferred without interruption, and the packet transmission number will change. While the new FAA is calculated, in a case when the selected server is completely different but the division ratio didn't change, the server will not sent the DM packet to the mirror server again. During the FAA recalculation, if some or all of the selected servers have been changed, the client finds out that the new mirror server ID is not the same as that of the original mirror server. Then he will send the TM (Terminate Message) to the new mirror servers and stop the file transfer. The newly selected mirror server will receive a new DM packet message (including some messages from the previous file transfer servers) and will transfer files in accordance with the new DM packet message.

b.

c.

Large file download simulation In the simulation, we create different sets of network environment with a random link to the client's router nodes in the routing topology. The client router nodes are linked by Twisted Pair and are randomly assigned bandwidths, trying to achieve the lowest delay time. In a network environment established with the client, 1GB of files will be downloaded respectively for the three kinds of protocols. This simulation was repeated 100 times, and each time the simulation experiments regenerated a new set of links to the client, then we aggregated and averaged the data. The results are shown in Figure 3, when the large file download time is the biggest for P-FTP, lower time for MFTP, and the minimal download time for DP-FTP it is the fastest among the three protocols. Moreover, with the increase of the file size, the gap between the performances of these three protocols also increases. This proves that if the client chooses the best mirror server according to the server efficiency of the transmission delay time to perform segmentation into sub-files, the overall

721

download will be speeded up, and the client waiting time reduced.

server will increase the ratio, so the download time increases. In our proposed DP-FTP, it is up to the client to choose the mirror server, so he can take a full advantage of the whole AS region and choose the mirror servers. Even if the number of clients increases to 10, the DP-FTP method adjusts to the network condition and selects the best mirror servers, so that the burden on the downloading network is balanced, and the time increase rate is very small. C. Multiple users download in interval time This simulation experiment was conducted in a manner identical with the experiment 2, except that the clients were generated at the interval of 10 seconds. In this simulation, an interval between clients was generated, to find out whether each protocol can efficiently allocate the mirror servers and download transfer work scattered in various mapping servers to increase the transmission efficiency. The result is shown in Figure 5. This simulation experiment is the same as experiment 2, but a new client was generated every 10 seconds, the previous experiment test whether protocol can be balanced for download, and if work can be assigned for download transmission in each respective server. In figure 4 we can see that when multiple users download interval increases, the situation is more stable than in figure 3, because the clients at intervals enter a network environment, so each could be assigned a suitable mirror server. Then, as the network environment changes the protocol adjusts the selected mirror server and assigns sub-file size to it. In these three experiments, no matter how large the file downloads are, or how many multiple users simultaneously download files, or at what intervals multiple users download files, when we compare the various network protocols, we can find that DP-FTP method is the fastest, its relative download time is faster than those of MFTP and P-FTP. DP-FTP is based on the network environment designed for optimal file download and achieves the most efficient allocation of servers and the best transmission performance.

Figure 3. Large file download simulation B. Multiple users download at the same time In this simulation we recreate the situation where multiple users simultaneously download files from the server environment, generate random links to the clients' router nodes and perform 1GB file downloads. After completion of the first simulation experiment, we use the same network environment and then generate a second client and his random routing node while retaining the first client. This simulation experiment gradually increases until the client number reaches 10. All clients request the download for 1GBytes file at the same time. After simulating an increase up to 10 clients, we create a different set of network environment and repeat the above steps, adding up to 10 clients to the first one. The result is a total of 10 sets of duplicate network environmental tests, and the data is aggregated and then averaged. The main purpose of our experiment is to compare how each of the three protocols works when the number of clients increases, and how long the download time with each protocol is. When two or more clients on the network environment make the request, each protocol must find the mirror server for individual clients at the same time and split the subfile size. The transport protocol is able to efficiently allocate the download, and the burdens will affect the overall transmission efficiency of the network environment and the client's download time. When at the same time the number of user increases, P-FTP curve with more than five users shows that the download time rapidly increases. We can see that the increased burden on P-FTP server will also increase the download time, because P-FTP server using a fixed calculation to select the mirror server which would link the client requests to the same mirror server, resulting in an increased burden. The MFTP has the flexibility to select the server, although its download process is more efficient than with P-FTP, it still must select a mirror server through an intermediary server. When the number of clients is greater than 7, the client selection of the duplicate

Figure 4. Simultaneous download by multiple users

722

VIII. ACKNOWLEDGMENT This work was supported by National Science Council, Taiwan (NSC 99- 2221- E- 324- 043).

REFERENCES
[1] [2]

[3]

[4]

Figure 5. Interval downloads by multiple users VII. CONCLUSION To transfer large files by networking is an important Internet application. In this study, the Distributed Parallel File Transfer Protocol, (DPFTP) is proposed for multi-users. The proposed method is a modified form of the Parallelized File Transfer Protocol (P-FTP). The proposed method finds the suitable mirror servers with the help of the client himself, and calculates the size of transmission sub-files for each mirror server found. The selection of the mirror servers depends on the network parameters. The client may change the mirror server when the situation of the network is different, such as there is another client requiring a transfer. According to the simulation results, the proposed method is suited for the multiusers in the parallelized file transmission.
[5]

[6]

[7]

[8]

[9]

[10] [11]

Stewart, R. and Metz, C. “SCTP: new transport protocol for TCP/IP,” IEEE Internet Computing, Vol. 5, No. 6. pp. 64-69 (2001). S. Sohail, S. Jha, and H. Elgindy, “Parallelized File Transfer Protocol (P-FTP),” Proc. IEEE Local Computer Networks Workshop High-Speed Local Networks, pp. 624-631, 2003.. Bankovic, Z., Stepanovic, D., Bojanic, S., and Nieto-Taladriz, O., “Improving network security using genetic algorithm approach,” Computers and Electrical Engineering, Vol. 33, pp. 438-451 (2007). Fu, Z. and Venkatasubramanian, N., “Directory Based Composite Routing and Scheduling Policies for Dynamic Multimedia Environment,” Proc. IEEE International Parallel and Distributed Processing Symposium, Vol. 9, pp. 398-410 (2004). J.-J. Liaw, C.-T. Wu and L.-H. Chang, “A Distributed File TransferProtocol based on P-FTP,” Proceedings of the Fifth IASTED International Conference on Communication Systems and Networks (AsiaCSN 2008), pp. 84-89, Langkawi, Malaysia, April 2-4, 2008. S. Sohail, S. Jha and H. ElGindy, “Parallelized File Transfer Protocol (P-FTP),” Proceeding of the 28th Annual IEEE International Conference on Local Computer Networks, October 20-24, 2003, Bonn, Germany, pp. 624-631. Chiung-Ta Wu, “The Distributed Parallelized File Transfer Protocols,” Master Thesis, Chaoyang University of Technology, Taiwan, 2008. Jean-Chrysostome Bolot. End-to-End Packet Delay and Loss Behavior in the Internet. In Proceedings of SIGCOMM 1993, pages 289{298. ACM SIGCOMM, August 1993. athchar: http://www.caida.org/tools/utilities/others/path/char/, June 2004. Network Simulator-2, http://www.isi.edu/nsnam/ns/, June 2004. R. L. Carter and M. E. Crovella, “Server selection using dynamic path characterization in wide-area networks,” in Proc. IEEE INFOCOM, Vol. 3, Kobe, Japan, Apr. 1997, pp. 1014–1021.

723