• from SourceForge.net (calitko-0.6.3-src.tar.gz)
• using Bazaar (bzr branch http://bzr.calitko.org/releases/calitko-0.6.3)

New in this source code release:

• Started writing automated acceptance tests.
• Started work on the HTTP implementation.
• Finally integrated some of the Generic components for use in the HTTP implementation.
• BitTorrent packets are generated using packet-gen.
• BitTorrent partial implementation for Tracker requests.
• Fixed many compiler warnings. Create a scripts which filters out the warnings we chose to ignore.

The following people have contributed to this Calitko release:

• Peter Dimov
• Petr Zemek
• Paul Martel

I’m happy to announce that I submitted my master thesis on Friday, September 28. The topic is “Unit-Testing towards a Specification: A Systematic Approach” and can be downloaded from:

http://bzr.calitko.org/developers/peter/MasterThesis.pdf

In the thesis I do a quick introduction to Test Driven Development (TDD) and Mock Objects. Then I present CalitkoMocks - the Mock Object framework I created for Calitko - which we use for most of out tests. I also introduce some test refactorings and show how specifications in form of sequence diagrams can be extracted from tests. In one of the chapters I present a step-by-step example showing the use of TDD with Mock Objects (CalitkoMocks), test refactorings, and sequence diagram generation. The example is done in the context of Calitko and as such should be extremely useful for any new Calitko member.

Older versions of some of the chapters are already published as separate manuals under http://trac.calitko.org/wiki/Manuals. Hopefully in the near future I’ll find some time to update these manuals or even convert my master thesis completely to the wiki format.

I hope you’ll enjoy the reading!

Peter

I’m planning on further developing the Http component. I will target the functionality of downloading a complete file with a single HTTP request. It will be interesting to see how the Protocol/Generics components would work for a non-packet oriented protocol.

We also have dependencies of BitTorrent and Gnutella on HTTP, so in this regard it also makes sense to focus on our Http implementation. Once we have some HTTP downloads working we could devote some energy on our download services and UI.

My initial idea is to create an HttpClientSession on top of a Generics::Session object (similarly to BitTorrent::Transfers::TransferSessionImpl). The actual Generics::Session object would be a Generics::GenericSession initialized with an Http::DataSerializer. Http::Header and Http::BodyChunk would derive from Generics::Data and HttpClientSession would represent HTTP request and response messages as a sequence of Http::Header HttpBodyChunk *.

I’d like to give the Http::DataSerializer a try… anybody interested in the HttpClientSession part?

Regards,

Peter

As a follow-up to Petr’s post I’d like to express my vision of Calitko. I’ll be looking forward to reading your views on our common aim and the way to achieving it. I would appreciate any feedback and ideas how to better communicate the message and aim of Caltiko Project on our own and other sites.

The ultimate aim of Calitko Project is to create an extensible software P2P platform. Calitko is both a framework and the application we build while developing the framework. The idea is not to include Calitko as a library to other projects, the idea for other projects to build upon Calitko. A suitable example I think is Eclipse. Calitko would provide core services which 3rdParty plugins could build upon to extend Calitko and provide new services.

Calitko would be of interest to developers fond of software technologies, which care about creating high quality software at all levels (down from source code up to the UI). It would be of interest to P2P researchers, who would be able to easily experiment with existing and new protocols because most of the tasks would already be handled by Calitko (for example, we already have packet-gen which allows us to describe packets in XML and have the C++ code generated).

The architecture of Calitko Project is quite simple - we have three layers of abstraction - Protocols, Services, UIs. We are currently doing work exclusively at the lowest layer (Protocols). If at the Protocols level we have the file-transfer and file-sharing protocols HTTP, FTP, Gnutella, BitTorrent, we could have a single FileDownloader service at the Services layer. Ideally, once we have the FileDownloader service we would be able to extend it to support infinite number of new protocols without many modifications to FileDownloader itself. Similarly, we could create a new service at the Services layer that uses HTTP or Gnutella for something different than file transfers, e.g. for media streaming. At the UIs layer we could have alternative UIs for each of our services.

I see the best way to proceed is bottom-up, i.e. from Protocols through Services to UIs. First we would concentrate on file transfers (both downloads and uploads), next we could move to instant messaging or distributed computations. Once we have a solid base of code and services and more and more new people join us, we would have an infinite set of possibilities - that’s what I love about software most - aside from CPU and memory, which are increasing steadily, the only limitation we have with software is our own imagination!

Best regards,

Peter

I agree with you that we should disable the calitko.pri subproject since we’re not working on it. However, I think that we should at first focus on some protocol implementation (BitTorrent seems to me as the best choice now) that will use our generic interface so we have something like a simple and working p2p “framework”. Then I’d suggest to re-open the calitko.pri project and try to refactor the application to use our “framework” with some basic generic functionality instead of the old Gnutella code. Then I’d try to add basic BitTorrent functionality (maybe as a some kind of a plugin?), step by step (as you wrote), so the application will be functional and up-to-date. After we have this, we could regularly add some functionality to the tester.pri and right then to the calitko.pri.

I know that it will be actually more difficult because I haven’t elaborated on this too much and I don’t know exactly what the application should do, but what I want to say is that I think that having a generic p2p “framework” is more important than having a more-or-less functional application.

Could you please describe your (concrete) ideas what the application actually should do (or you’re planning it should do)? I’d really appreciate it since I think it will bring more light into my thinking about this topic.

Have a nice day,

Petr

I’m just back from a vacation and now I’m looking forward to doing some Calitko work again!

I was thinking recently that in at least in the last 6 months we have done almost no progress in the calitko.pri subproject. The application that it builds mainly runs the code in Gnutella and UIs. The tendency is anyway to refactor all of Gnutella to Protocols/Gnutella reusing the components in Protocols/Generics, so I suggest to temporarily disable the calitko.pri subproject. This way people checking the app every few months would not think we are doing no progress at all.

We should try to test & implement a complete set of classes implementing certain functionality which could be integrated in a new app project. For each new release we should target at testing and implementing new functionality in tester.pri and integrating it in the the new calitko.pri. Since each of us only has limited time to work on the project we should work together on a common short term target in order to get more stuff integrated in the app and less code only covered by tests.

What do you guys think?

Regards,

Peter

Yes, we should remove them, but you could filter all such warnings from the Gnutella package.

Ok, I’ve added them to the ignore list for now.

value_ is non-pointer member. Could you look more into that please?

The official Intel compiler docs says about this warning:

Use delete on pointer members in destructors. The compiler diagnoses any pointer that does not have a delete.

And this is what one would expect, but as I wrote, this warning appears also in classes with no pointer members at all… I haven’t found anything about this on the Internet neither.

I’d suggest that we tag all classes with one of: REFERENCE_OBJECT, VALUE_OBJECT, STATIC_HELPER. Maybe we could also create a custom Doxygen command (seems to be possible to be defined in Doxyfile) and tag also the docs.

I agree!

./Gnutella/LocalPeer.h(36): warning #64: declaration does not declare anything
class UIs::Searching::SearchModel;

What’s the exact meaning of this warning? That the forward declaration is unnecessary because the the class is fully specified already?

No, in this case it means that class UIs::Searching::SearchModel was already declared in the imported UIs/Searching/Namespace.h header. The warning itself (it’s “name”) is a little bit awkward, though.

Declaration of “some variable” shadows a member of ‘this’

We should enforce the memberName_ naming convention

Yes, I agree, especially when not using PrivateData class as a private data storage.

Please fix as many warnings as you can in your branch and I’ll merge all your revisions at once.

Maybe you could also update your calitko-dev branch and also fix all remaining BitTorrent warnings. I’ll do the same to fix the warnings in Utils::ObjectDispatcher.

Ok, I’ve fixed nearly all warnings (except these in the Gnutella package) - the latest revno is 219. I’ve fixed them in my calitko-dev branch, so I could fix also nearly all warnings in the BitTorrent package. I’ve also merged my calitko branch, so you don’t have to merge it - you can merge only my dev branch.

I’ve also updated our ignored warnings lists - I’ll soon update the wiki documentation to be up to date.

You can take a look into the attached file for a short list of warnings that still need to be fixed, nevertheless I’m really glad the we managed to fix nearly all warnings ;). There are still some problems, though (see TODOs in linux-icc.txt file).

Regards,

Petr

Great post! I agree with your suggestions, I’ll only comment some of the warnings.

using-declaration ignored — it refers to the current namespace
Appears only in the Gnutella package (79 total).

./Gnutella/NodeInfo.h(36): warning #780: using-declaration ignored — it refers to the current namespace
using Gnutella::NodeType;

These using declarations are superfluous, so we should remove them.

Yes, we should remove them, but you could filter all such warnings from the Gnutella package.

Effective C++ Item 6 missing delete of member pointer member “Some variable” in destructor
Nearly everywhere (777 total).

Protocols/BitTorrent/Bencoding/BString.h(58): warning #2017: Effective C++ Item 6 missing delete of member pointer member “Protocols::BitTorrent::Bencoding::BString::value_” in destructor
{}

I don’t really get (understand) this warning - it appears also on places where there is no pointer member variable… I’d understand it if it was only in dtors of classes with some pointer member variable.

value_ is non-pointer member. Could you look more into that please?

Effective C++ Item 11 declare a copy constructor and assignment operator for PrivateData
Gnutella, Http and CalitkoMocks mostly (734 total).

./Http/HeaderReader.h(65): warning #2021: Effective C++ Item 11 declare a copy constructor and assignment operator for Private
} p;

I first thought that this warning was useless, but then I saw that in classes where we don’t use QSharedData (or some other class that have defined copy ctor/assignment operator) for storing private data we don’t have our own copy ctor/assignment operator defined, but there are pointers in the PrivateData classes. This could lead to problems (copy ctor/assignment operator generated by a compiler performs only shalow copy, not deep copy). My suggestion is to either use our REFERENCE_OBJECT macro to disable copy ctor/assignment operator, or use QSharedData instead of a struct or define our own copy ctor/assignment operator.

I’d suggest that we tag all classes with one of: REFERENCE_OBJECT, VALUE_OBJECT, STATIC_HELPER. Maybe we could also create a custom Doxygen command (seems to be possible to be defined in Doxyfile) and tag also the docs.

Effective C++ Item 12 field member “Some class member” not initialized (preferable to assignment in constructors)

g++ reports that too - all of them should be fixed now (except for Protocols/BitTorrent and Utils/ObjectDispatcher which are probably fixed in other branches already).

Declaration does not declare anything
./Gnutella/LocalPeer.h:36.

./Gnutella/LocalPeer.h(36): warning #64: declaration does not declare anything
class UIs::Searching::SearchModel;

What’s the exact meaning of this warning? That the forward declaration is unnecessary because the the class is fully specified already?

conversion from “int” to “uchar={unsigned char}” may lose significant bits
Gnutella, UIs, Utils::Uri (115 total).

./Gnutella/Packets/QueryHits.h(160): warning #810: conversion from “int” to “uchar={unsigned char}” may lose significant bits
uchar numberOfHits() const { return p.resultSet.size(); }

We should check whether there could be no range problems.

g++ does not seem to report that. We should probably assert that p.resultSet.size() < = 0xFF. If we use a static_cast the warning should go away.

Declaration of “some variable” shadows a member of ‘this’

We should enforce the memberName_ naming convention.

function “QDialog::done(int)” is hidden by “UIs::ManagePacketsDialog::done” — virtual function override intended?

g++ reported that too - I’ve fixed it in calitko,141

Please fix as many warnings as you can in your branch and I’ll merge all your revisions at once.

Maybe you could also update your calitko-dev branch and also fix all remaining BitTorrent warnings. I’ll do the same to fix the warnings in Utils::ObjectDispatcher. We’ll very soon have clean builds ;-)

Best regards,

Peter

]]> http://www.calitko.org/source-talk/937/feed/ http://www.calitko.org/source-talk/936 http://www.calitko.org/source-talk/936#comments Sat, 18 Aug 2007 16:48:43 +0000 Petr Zemek Source Talk http://www.calitko.org/source-talk/936 Dear all,

I’ve recently added extra warnings for ICC (Intel C/C++ compiler) and compiled my latest calitko branch (developers/s3rvac/calitko,142) with this compiler (using our compilation.py script). There are some warnings which I’d like to discuss (I’ll follow the following layout: “name” of the warning, places of appearance (example[s]), suggestion how to fix it or why we should ignore it). The total count of warnings is only orientation.

using-declaration ignored — it refers to the current namespace
Appears only in the Gnutella package (79 total).

./Gnutella/NodeInfo.h(36): warning #780: using-declaration ignored -- it refers to the current namespace
using Gnutella::NodeType;


These using declarations are superfluous, so we should remove them.

Effective C++ Item 1 prefer const and inline to #define
Everywhere (18967 total).

Protocols/BitTorrent/Bencoding/BString.h(24): warning #2012: Effective C++ Item 1 prefer const and inline to #define
#define PROTOCOLS__BIT_TORRENT__BENCODING__B_STRING_H


Since this warning appears with each preprocessor directive, it’s useless. I think we all know that macros are “evil”, but there are situation where they’re handy and can’t be replaced with const/inline. My suggestion is to ignore this warning.

Effective C++ Item 4 prefer C++ style comments
Gnutella,Http and UIs mostly (360 total).

Http/Testing/HeaderTest.cpp(204): warning #2015: Effective C++ Item 4 prefer C++ style comments
CPPUNIT_ASSERT (m_p->isValid() == false ); /* Does not work */
--
UIs/Searching/SearchModel.h(40): warning #2015: Effective C++ Item 4 prefer C++ style comments
bitrate("8 bps"), /*parent(NULL), */hosts(), showEmphasized (false)


There are no problems, this is just a matter of style, so we could some of them replace with “C++” comments, but I’d suggest to ignore these.

Effective C++ Item 6 missing delete of member pointer member “Some variable” in destructor
Nearly everywhere (777 total).

Protocols/BitTorrent/Bencoding/BString.h(58): warning #2017: Effective C++ Item 6 missing delete of member pointer member "Protocols::BitTorrent::Bencoding::BString::value_" in destructor
{}


I don’t really get (understand) this warning - it appears also on places where there is no pointer member variable… I’d understand it if it was only in dtors of classes with some pointer member variable.

Effective C++ Item 11 declare a copy constructor and assignment operator for PrivateData
Gnutella, Http and CalitkoMocks mostly (734 total).

./Http/HeaderReader.h(65): warning #2021: Effective C++ Item 11 declare a copy constructor and assignment operator for Private
} p;


I first thought that this warning was useless, but then I saw that in classes where we don’t use QSharedData (or some other class that have defined copy ctor/assignment operator) for storing private data we don’t have our own copy ctor/assignment operator defined, but there are pointers in the PrivateData classes. This could lead to problems (copy ctor/assignment operator generated by a compiler performs only shalow copy, not deep copy). My suggestion is to either use our REFERENCE_OBJECT macro to disable copy ctor/assignment operator, or use QSharedData instead of a struct or define our own copy ctor/assignment operator.

Effective C++ Item 12 field member “Some class member” not initialized (preferable to assignment in constructors)
Gnutella, CalitkoMocks and generated files mostly (220 total).

Gnutella/Bootstrapping/NodeCache.cpp(81): warning #2022: Effective C++ Item 12 field member "Gnutella::Bootstrapping::NodeCachePrivate::availabilityList" not initialized (preferable to assignment in constructors)
NodeCachePrivate() : nodeTable(), completeList() {}


This warning is similar to the g++ “…should be initialized in the member initialization list”, so we should ignore it if it’s in some PrivateData class and fix it otherwise (most of these warnings are already fixed).

NULL reference is not allowed
CalitkoMocks (6 total).

./Utils/CalitkoMocks/BasicTypes.h(135): warning #327: NULL reference is not allowed
T & DefaultValue <T &>::value = *reinterpret_cast <T *> (0); // HACK!!!


Peter will fix this (we’ve already discussed this one).

Declaration does not declare anything
./Gnutella/LocalPeer.h:36.

./Gnutella/LocalPeer.h(36): warning #64: declaration does not declare anything
class UIs::Searching::SearchModel;


Superfluous, remove it (there is even written: “// get away with these:”).

conversion from “int” to “uchar={unsigned char}” may lose significant bits
Gnutella, UIs, Utils::Uri (115 total).

./Gnutella/Packets/QueryHits.h(160): warning #810: conversion from "int" to "uchar={unsigned char}" may lose significant bits
uchar numberOfHits() const { return p.resultSet.size(); }


We should check whether there could be no range problems.

Parameter “some variable” was never referenced
Gnutella, UIs and Utils (25 total).

UIs/DownloadProgressBar.cpp(57): warning #869: parameter "start" was never referenced
void DownloadProgressBar::removeRequestedRange (int start, int end)


This is similar to the g++ “unused parameter” warning - we have already discussed this one (we should keep this warning since there is mostly some work needed).

Variable “variable name” was declared but never referenced
Variable “variable name” was set but never used
CalitkoMocks mostly (16 total).

Utils/CalitkoMocks/Testing/CallDriverTest.cpp(119): warning #177: variable "expectationDriver" was declared but never referenced
expectationDriver = driver.willCall (pair);


This is similar to the g++ “unused variable” warning - see the previous warning.

Controlling expression is constant
Protocols/BitTorrent/Packets/BadPacket.cpp (6 total)
Utils/Uri.cpp (1 total)

Protocols/BitTorrent/Packets/BadPacket.cpp(54): warning #279: controlling expression is constant
Q_ASSERT (false && "Modifying header fields of BadPackets not allowed!");


We’ve agreed to change the form of these assertions to Q_ASSERT (false); // Modifying header fields of BadPackets not allowed!. I think it should be fixed in the Peter’s dev branch already.

Declaration of “some variable” shadows a member of ‘this’
Protocols package, UIs package and CalitkoMocks (15 total)

Protocols/Generics/TcpSocketBuffer.cpp(167): warning #1944: declaration of "readSize" shadows a member of 'this'
the shadowed declaration is at line 93 of "Protocols/Generics/TcpSocketBuffer.h"
int readSize = 0;


We should check all of these and give these variables another name (it could be misleading).

Declaration hides variable “variable name”
Gnutella package, Http/Header.cpp and Protocols/Generics/PacketSerializer.cpp (8 total).

Protocols/Generics/PacketSerializer.cpp(61): warning #1599: declaration hides variable "header" (declared at line 57)
QByteArray header = transport->read (headerLength);


This is more or less the same warning as the previous one (we should give these variables another name, too).

function “QDialog::done(int)” is hidden by “UIs::ManagePacketsDialog::done” — virtual function override intended?
UIs/PacketDumpTreeView.h:55 (1 total)

UIs/PacketDumpTreeView.h(55): warning #1125: function "QDialog::done(int)" is hidden by "UIs::ManagePacketsDialog::done" -- virtual function override intended?
void done();


If virtual function override was intended, we should change the signature of that function. If not, we should change the name of the function.

These are all warnings that were produced by the Intel compiler (uff :-)). Feel free to write down your suggestions! You can also check the compilation log (see attached file).

Regards,

Petr

Attached Files:

icc-output.zip 398K

]]> http://www.calitko.org/source-talk/936/feed/ http://www.calitko.org/source-talk/935 http://www.calitko.org/source-talk/935#comments Sat, 18 Aug 2007 08:15:26 +0000 Petr Zemek Source Talk http://www.calitko.org/source-talk/935 Hi Peter,

I think the problem is caused by the two versions of getNodes(). There is a template and non-template one. It seems that the icc compiler prefers the template version and g++ prefers the non-template one. I think a simple cast of the last argument would remedy the situation:
[snip]

Yes, that worked. I’ve fixed it in my calitko branch, rev142. However, I thought that if a compiler have to decide whether to use a template or a non-template function, it must choose the non-template one… That is why I was surprised when I found this error on the Intel compiler. But maybe I’m just missing something.

I have also done some updates in the rev 141, most of them are related to the ICC compiler, so you can merge these changes to the main branch, too (compilation went OK with this version on both compilers - g++ and ICC). I’ll soon write a post about extra compilation warnings produced by the Intel compiler (with attached logs), so we can discuss them and update the (new) linnux-icc.txt ignore list (and also wiki pages).

Btw. I’m on my way to fix all warning (except the ones in BitTorrent which you have probably fixed already) and I’ll write a post once I’ve done that! The filtering script works like a charm for me :-) !

Great!

Have a nice day,

Petr

]]> http://www.calitko.org/source-talk/935/feed/