Forums : Technical Issue Help

the project wasn't updated for the last three months; url is still the same

Danny Götte almost 16 years ago

Hi Danny,

I restarted the job for ReactOS. It will take a while to finish; the repository has many commits to pull.

Andy Verprauskus almost 16 years ago

Hi,

it failed again.

Danny Götte almost 16 years ago

Hi Danny,

Our system had rescheduled the work by the time I went to check on it.

Downloading the commits is taking a long time; I'd guess that some of the commits have a lot of changes/files in them. Does the repository include packages or other binary file changes frequently?

A related question: do you know if the repository supports svnsync? That might work better. As it stands, it'll probably be a few more failures and several weeks of downloading on our part to catch up on ReactOS if the commit downloading continues to be slow.

Thanks

Andy Verprauskus almost 16 years ago

While we avoid big binaries in our repository and won't ever store release packages or stuff like that in it, there could be some big changes in the range you're downloading. (e.g. vendor imports)

Our server is currently a SVN 1.5.x one, thus svnsync is fully supported.

Colin Finck almost 16 years ago

Thanks Colin.

I've started a second enlistment for ReactOS and forced it to use svnsync.

That might speed things up. I'll keep an eye on this today.

Andy Verprauskus almost 16 years ago

now ohloh fails at counting the lines :(

Danny Götte almost 16 years ago

Hi Danny,

We are getting an inconsistent line ending style error when trying to view a file from this repository:

$ svn cat -r 2 svn://svn.reactos.org/reactos/trunk/reactos/lib/ntdll/genntdll@2
svn: Inconsistent line ending style

Can this be fixed? If we can't run the svn cat command for every file in the repository, we can't run our code analysis.

Thanks,
Robin

Robin Luckey almost 16 years ago

We certainly can't change decade-old revisions in our repository.
Besides, this somehow used to work before and if you really run it for every file in the repository, you will surely get into problems with other binary files.

If these svn cat commands are run by a script, it should be possible to check the return value of that command and just skip a file in case of a non-zero value. That would also save you from hassles with similar repositories.

Colin Finck almost 16 years ago

Hi Colin,

Ohloh has two strategies for downloading Subversion history.

One strategy is to convert the repository to Git. We do this by checking out each revision in turn, and as each revision is checked out, we check it in to a new Git repository. As you might imagine, this is extremely slow, but it is also extremely robust. Our original report for ReactOS was made this way.

Our newer, preferred strategy is to use svnsync to create local mirror. This is the most efficient method. However, it's not as robust: it relies on a very clean, error-free repository and a server that permits us to use svnsync.

Because we had been having a lot of performance problems downloading ReactOS using our initial strategy, Andy tried creating a new download using the svnsyc strategy. Although the download was very fast, it didn't work for us because of consistency problems in the repository.

So I've reverted us back to the old strategy. It's going to be slow, and I can't guarantee that it will finish. I'll keep an eye on it this week.

You can in fact make changes to decade-old revisions, but it's not easy and I certainly wouldn't expect you try. However, some folks are crazy enough to do it, so I threw out the specific error just in case you are interested.

The problem with skipping any file that returns a non-zero result code is that more often than not there is a systemic problem in the repository that will result in a very bad report if we keep going. There are some types of errors that our system does ignore, but in this case we don't keep going because where there's smoke, there's often fire.

As it turns out, we don't see the inconsistent line-ending style error very often. After tens of thousands of repositories, I can only recall seeing this one a dozen times.

So we're back to our old, slower strategy. I'll keep an eye on it and hopefully we'll make some good progress this week.

Thanks,
Robin

Robin Luckey almost 16 years ago

Thanks for the long and detailed answer.

If Git can be helpful here, you may try our own Git mirror of the SVN repository (http://git.reactos.org/?p=reactos.git;a=summary), which was set up some months ago.
Although we primarily use SVN and will probably continue using it for a long time, the Git repo is synchronized with the SVN one every 10 minutes, so it should be quite up to date.

Best regards,

Colin

Colin Finck almost 16 years ago

A git mirror would be fabulously helpful.

From the web URL you provided, I can't figure out the Git-compatible URL. What URL do I feed to git?

Robin Luckey almost 16 years ago

Please try git://git.reactos.org/reactos.git/

Colin Finck almost 16 years ago

I've started a report based on the Git repository -- this should work out much better.

Thanks,
Robin

Robin Luckey almost 16 years ago