emule ZZUL TRA TL

Emule ZZUL TRA Torrent Like (TL) mod
lastest version: 0.6f

Files

Fortunately there are a lot of good free file hosts (with no waiting time or limited number of acces/day) for open and free data. So i will use it along with edonkey (emule) network1.
Update: check assembla repositories!

Executables

The executables, for some "critical options", are already set, if you aren't expert please leave extended options (or .ini options) unchanged.

Sources

Yes, also assembla decided to change the URLs and things, that may be there, aren't reachable anymore.

Introduction and News

This heavily mod is based on emule ZZUL TRA (a huge thanks to morph4u, zzul, emule team and others contributors of the ZZUL code). Recently i have read some papers of companies that make routers and other "serious" network hardware2; these papers report analysis of data exchanged by the nodes of some ISPs and gathered by special routers that can do statistics about the data exchanged. The consequences of these analysis are:

  • p2p cover about the 50~65% of the world internet traffic.
  • the major protocol is torrent, that cover more than 50% of the p2p traffic, and the rest is not covered only by edonkey network (emule). Edonkey share about the 10~25% of the p2p traffic.

Since whatever p2p client exploits well the upload bandwidth of the user, we can suppose that torrent has a lot more users than emule. Anyway torrent has major drawbacks3 (imho):

  • the bandwidth is focused, by the official software itself, only on few active torrents (default, If I Remember Correctly, 5).
  • The official client itself remove from sharing completed or not active torrents under some conditions (for example, there are too many torrents under download).
  • The client upload bandwidth is mostly assigned to peers that have the best value for the difference: data sent to this client - data received from this client.

With these features, torrent can satisfy user's needs for a wide range of files. These files get a new request in relatively short time intervals, else they will disappear from the network. Emule, conversely, shares a lot of files without remove them from sharing, so a file owned by some clients can be always reached. This is much better for file diversity, and file diversity is needed because each user has different tastes.
Then emule is more democratic than torrent from the point of view of files, torrent is more oligarchic.

So why this mod? Because the people reward more the download speed instead of file diversity. Torrent, see its main features explained above, is practically a TFT p2p client. So the main goal of this mod is: modify emule so it can work almost as a TFT client (the "almost" will be explained below), then emule can satisfy more users, this means more users will use emule and file diversity won't be lost (as it happens in torrent).

Changelog

See changelog.

Features explained

A lot of features are inherited from emule ZZUL TRA that is, in my opinion (imo), one of the best emule mod4. Changes in ZZUL TRA TL are simple in the code but very effective for the client behaviour.

TFT Behaviour

TFT behaviour is achieved when PayBackFirst (PBF) is enabled (see Options—>Extended). PBF works in this manner: if the remote client that ask data is correctly identified, support the obfuscation protocol and has sent to us more data than we send back to him (specified by "PayBackFirstLimit"5 ), then PBF is enabled. The client will jump in the top of the waiting queue, thus minimizing its waiting time to obtain an upload slot.
Except clients that ask powershared files, any "normal" (i.e. that not ask powershared files) clients can overtake a PBF client in the waiting queue (in fact powershare and PBF clients share the same priority class).

Why is TFT not bad for the edonkey network and file diversity?

The idea is: all the starting nodes in the network can't do always TFT, sooner or later they stop downloading from the same users or start to request files to other clients, so new clients won't wait too long.

For example: worst case
There are three nodes A,B,C.
Each nodes has credits to the other to activate the PBF threshold and all of these nodes ask the same file.

So, the starting configuration is:
A asks a file to B, B to C, C to A.

We suppose that there is only one upload slot, so no new clients are allowed.

Sooner or later there is a time instant that A doesn't satisfy PBF conditions on B, the same happen to B on C and C on A.
Now the situation is reversed, because A has gather enough credits on C, so it activate the PBF on C, same for B on A and C on B.

This scenario will end only when all clients complete the file, after there are 2 options:
- a similar scenario begins because A,B,C request similar files each other.
- A,B,C are no more interested each other. So new clients can get upload slots.

Note that Torrent use a more "clever" TFT, avoiding upload to clients that don't have interesting pieces, and this is really bad.

Notes for me here

It's better to not include SUQWT in the code

(note4me: because combo TFT plus huge waiting time)

Only Punitive ClientAnalyzer

(note4me: Also no powershare for bad CA clients that don't satisfy the PayBackFist condition)

Questions

( For several reasons i think that contributes should be written in the same place of the topic. For example to avoid dispersion and loss of content if the other place is shut down. For example all my contents on specialmods' forum are lost :/ . It's like a (tiny) part of my life is gone )

Disclaimer: i won't write any proof of my concepts, because it requires times and moreover a lot of people will simply say "oh, it is sounds like a nonsense for me, it's not a proof" (in other words: i don't think that with the majority of others developers there are chances for a constructive debate). I just give an idea of my point of view on various emule elements.

Whats is the point of your Mod?

I love the philosophy behind emule "give a chance to all files/client to get the file, and they will wait patiently" but anyway a great idea is useless if is not used. So i did a bit of research, and the results were: the great part of internet traffic, thus the bandwidth on internet, is used by the torrent protocol. And this is logic, because people are quite eager, they don't think about the common good, but about their good. So they want the file as soon as possible.

But the torrent protocol has a costly tradeoff: the clients focus bandwidth on few files to achieve huge speeds, so torrent compilations (and not only single files) that are not so popular dies quickly and are no more reachable. Moreover clients will disconnect after getting the file, so there are plenty of private trackers that enforce ratio to avoid leeching (that is easily achievable with the default client!!!).
Then torrent have a great pro: good speeds; but two great cons: small file diversity and high dependencies from centralized servers to achieve a democratic sharing plus search capability.

Emule doesn't have these problem (there is a small problem about searches, since now each users have tons of files that kad can't index concurrently, but there are ideas to solve this problem) except the problem of speed.

So, since an emule client knows nothing about the network, what can he do to foster democracy and speed? Rewards users that show him that are good users (same identifier for long time or they upload a lot to the client or at least they take enough, giving nothing, but not so much from the client). But PayBackFirst (reward who give us more than what we give to it) should be enabled after the completion of the file? No. Why?
Because the probability that two clients can be interested each other for more than a file is really low, so they should reward each other ASAP on whatever file.

But doing so there is a Tit for that behavior. Yes, but is not a problem since we should see facts (lol, for me are facts at least). A lot of people are lazy, i mean: they can disconnect emule after the file is in their hands, but the probability that they will unshare that file is really low, so in the next connection that file will be available.

Yeah but is a false availability, since the client will done continuously tit for that with new clients. False. There are at least two limits: first 1:3 ratio, he can download but not so fast in current session (since statistics about all session can be modified manually) and his download is fixed to how much he uploads. Then he should wait (since files, normally, are huge) at constant download speed6 giving to the network relatively enough bytes. So who has credits towards him can use the PayBackFirst, an a delayed TFT (TFT that works between different sessions) is not a TFT at all.
Moreover the client, when he (i see it as a user) connects to new clients, has no credits to them so he should wait and in the meanwhile he can give back to the network what he picks in previous sessions.

In other words: TFT coupled with other limits can catch the user that see "speeds" and think (oh, the download is going) but in the same time force him to give bytes to other peers, preserving, in the long run: file diversity and real peer to peer (thanks to KAD that is great) because emule doesn't need servers.

To demonstrate this i should do a huge emule simulator, but then one can say "the probability distribution of clients that search new files is not real!!! Your simulator is useless", and this discourage me. In fact we have no real statistics about emule users, i can infer the "average user behavior" from lot of discussion on the web and a lot of observations of clients in the queue, but anyone can dismiss these inferences.

What about Sub Chunk transfer feature and a new limit for PayBackFirst ?

(The ability to send bytes of a chunk that is incomplete)
It's great, it avoid to wait chunk completion that can need even 40 minutes (@4kb/s) to be completed, so these bytes can circulate in the network earlier. But anyway the threshold to activate PayBackFirst should be enough , for example at least a chunk of difference (between our uploads to that client, and uploads of that client to us).

The goal of my System is to balance q/file UL based on availability by using CA. Unfortunately your method removed that part of CA. Thats my primary question for you: wft did you do that?

I suppose that the availability of a file is proportional to requests of that file: more the requests, more the sources. Moreover in my mod there is a sort of "one chance per file", i mean: the queue is ordered by "last time that someone gets some bytes of that file" and after by "client waiting time". Then a file that was uploaded some minutes ago has a low score, and then all the clients that request that file have a low score, while files that are never uploaded in the current session have the (relative) maximum score, that is slightly modified adding the score given by the waiting time in the queue of clients.

The speed of "recovering score" is modified by priority (it's a partial work actually, July 2013). If a file is in release, it recovers its score ten time faster than a file with normal priority, so in the long run will be shared ten times more. But obviously i suppose that files in release are few (else there is no difference between files, if they are a lot with same high priority. With no difference it's like all files have the normal priority or low priority or whatever), so in the meanwhile a lot of files with lower priority will be shared.

This to improve a bit the file diversity.

In the future i want to add the "sharing ratio" value. After a file is shared at least X times is size7, then it drops the priority to a lower level: if it is normal, it drops to low. Then it drops to very low.

Why are you on hold?

In fact i'm on hold because i want to, when i pick again emule:

  • Explain the philosophy of my mod and each feature that it has (not in detail, but at least as idea). No one developer has done it (and the community has explained only basics of emule) and it's bad because users have a lot of doubts and less chances to talk about the code (i'm still one of these);
  • Implement ideas from me or from others users (Feature requests sections) or mods (i.e: mods often cited about a feature) that i think are good for the network. I mean if the network will be populated only by my mod, it should behave better to achieve some conditions (else, if there is only my mod and things goes worse, my mod is bad). After the implementation and testing, explain these features as i said in the previous point.
  • Maybe, instead of implement new features on current emule for windows, it's better to spent a lot of time to port it on a language that is highly portable. Jemule (on java) it's quite good but i think also to mobile computers (tablets, smartphones and so on). Two good languages are python or lua. Because emule doesn't need a fast CPU (it works really well, with a huge queue an a lot of files, with a x86 @ 200mhz, like a pentium II 266mhz) while it needs a bit of ram (due to buffers and some other things) at least 256 mbyte of free ram. In technical terms is an "I/O bounded" software. With a porting (that require a lot of time) we can use emule with windows, linux, mac, android, iOs, symbian, windows phone, windows mobile, etc. Moreover, without needs of huge updates. For example Stulle is using often the last visual studio and his emule needs always the last .net framework. And this is really bad, since a computer with windows 2000 or linux with wine can't execute it. Open software should be executed on different platforms.
    Finally lua or python are much more simpler languages than c++ (even if not so efficient like c++, but this is obvious because they are interpreted to run everywhere, nevertheless with a I/O bounded software there are no problems of efficiency), then more people can do modifications.

    After that port, i'll implement ideas as i said previously.

Why do you not remove limitation of download speed? (ratio)

Yeah, you have right. If you find a good source with good upload you will be limited. But ratio (1:3, 1:14; 1:0.5; whatever; i picked 1:3~4 only because it is really common) force you to wait to finish the file. So if you wait more, you upload more and since total amount of upload on the network is equal to total amount of download of the network, this global value can increase (because upload won't be limited, only download).

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License