Topics

boost unit tests

Joshua Ulrich
 

Hi all,

I was able to successfully build the boost unit test framework using
msys1.0 and Rtools35. From the msys prompt and within the boost
source directory, I ran:

BOOST_SRC=/e/build/src/boost_1_68_0
BOOST_LIB=/e/build/boost
bootstrap.sh gcc
b2 -j6 -a --with-test --prefix=$BOOST_LIB address-model=64
--layout=tagged install
# TODO: build the framework outside the source directory (recommended for boost)

Then configure QuantLib with the following options:

--with-boost-include=$BOOST_SRC --with-boost-lib=$BOOST_LIB/lib

QuantLib test suite is currently running... I used:

/e/build/QuantLib/lib/x64/bin/quantlib-test-suite --log_level=message \
--build_info=yes --result_code=no --report_level=short

I'll follow up once the tests finish. I expect them to take at least
an hour, maybe two.

Best,
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com

Dirk Eddelbuettel
 

On 1 December 2018 at 14:06, Joshua Ulrich wrote:
| Hi all,
|
| I was able to successfully build the boost unit test framework using
| msys1.0 and Rtools35. From the msys prompt and within the boost
| source directory, I ran:
|
| BOOST_SRC=/e/build/src/boost_1_68_0
| BOOST_LIB=/e/build/boost
| bootstrap.sh gcc
| b2 -j6 -a --with-test --prefix=$BOOST_LIB address-model=64
| --layout=tagged install
| # TODO: build the framework outside the source directory (recommended for boost)
|
| Then configure QuantLib with the following options:
|
| --with-boost-include=$BOOST_SRC --with-boost-lib=$BOOST_LIB/lib
|
| QuantLib test suite is currently running... I used:
|
| /e/build/QuantLib/lib/x64/bin/quantlib-test-suite --log_level=message \
| --build_info=yes --result_code=no --report_level=short
|
| I'll follow up once the tests finish. I expect them to take at least
| an hour, maybe two.

Wicked. Thanks for setting that up.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

Joshua Ulrich
 

On Sat, Dec 1, 2018 at 2:09 PM Dirk Eddelbuettel
<@eddelbuettel> wrote:

On 1 December 2018 at 14:06, Joshua Ulrich wrote:
| Hi all,
|
| I was able to successfully build the boost unit test framework using
| msys1.0 and Rtools35. From the msys prompt and within the boost
| source directory, I ran:
|
| BOOST_SRC=/e/build/src/boost_1_68_0
| BOOST_LIB=/e/build/boost
| bootstrap.sh gcc
| b2 -j6 -a --with-test --prefix=$BOOST_LIB address-model=64
| --layout=tagged install
| # TODO: build the framework outside the source directory (recommended for boost)
|
| Then configure QuantLib with the following options:
|
| --with-boost-include=$BOOST_SRC --with-boost-lib=$BOOST_LIB/lib
|
| QuantLib test suite is currently running... I used:
|
| /e/build/QuantLib/lib/x64/bin/quantlib-test-suite --log_level=message \
| --build_info=yes --result_code=no --report_level=short
|
| I'll follow up once the tests finish. I expect them to take at least
| an hour, maybe two.

Wicked. Thanks for setting that up.
Happy to. I want to find the source of this bug!

My time estimate was way off. Unit tests ran in ~20 minutes and
everything passed.

I'm currently trying to run the "windows_bug_redone.cpp" example, but
I'm having trouble linking. Here are the commands I'm running, and
the linking error.

g++ -g0 -O2 -std=c++11 -m64 -I./src/QuantLib-1.14 -I./src/boost_1_68_0 \
-L./QuantLib/lib/x64/lib -lQuantLib -c windows_bug_redone.cpp -o
windows_bug_redone.o
g++ -static -o windows_bug_redone windows_bug_redone.o
-L./QuantLib/lib/x64/lib -lQuantLib

windows_bug_redone.o:windows_bug_redone.cpp:(.text$_ZNK8QuantLib27CPICapFloorTermPriceSurface7maxDateEv[_ZNK8QuantLib27CPICapFloorTermPriceSurface7maxDateEv]+0x1d):
undefined reference to `QuantLib::Date::advance(QuantLib::Date
const&, int, QuantLib::TimeUnit)'
windows_bug_redone.o:windows_bug_redone.cpp:(.text$_ZNK8QuantLib27CPICapFloorTermPriceSurface7minDateEv[_ZNK8QuantLib27CPICapFloorTermPriceSurface7minDateEv]+0x1c):
undefined reference to `QuantLib::Date::advance(QuantLib::Date
const&, int, QuantLib::TimeUnit)'
windows_bug_redone.o:windows_bug_redone.cpp:(.text$_ZN8QuantLib27CPICapFloorTermPriceSurface13checkMaturityERKNS_4DateE[_ZN8QuantLib27CPICapFloorTermPriceSurface13checkMaturityERKNS_4DateE]+0x3a):
undefined reference to `QuantLib::Date::advance(QuantLib::Date
const&, int, QuantLib::TimeUnit)'
windows_bug_redone.o:windows_bug_redone.cpp:(.text$_ZN8QuantLib27CPICapFloorTermPriceSurface13checkMaturityERKNS_4DateE[_ZN8QuantLib27CPICapFloorTermPriceSurface13checkMaturityERKNS_4DateE]+0x7b):
undefined reference to `QuantLib::Date::advance(QuantLib::Date
const&, int, QuantLib::TimeUnit)'
windows_bug_redone.o:windows_bug_redone.cpp:(.rdata$_ZTVN8QuantLib15AbcdAtmVolCurveE[_ZTVN8QuantLib15AbcdAtmVolCurveE]+0xd0):
undefined reference to `non-virtual thunk to
QuantLib::AbcdAtmVolCurve::update()'
windows_bug_redone.o:windows_bug_redone.cpp:(.rdata$_ZTVN8QuantLib15AbcdAtmVolCurveE[_ZTVN8QuantLib15AbcdAtmVolCurveE]+0xe0):
undefined reference to `non-virtual thunk to
QuantLib::AbcdAtmVolCurve::performCalculations() const'
windows_bug_redone.o:windows_bug_redone.cpp:(.rdata$_ZTVN8QuantLib23FittedBondDiscountCurveE[_ZTVN8QuantLib23FittedBondDiscountCurveE]+0xb8):
undefined reference to `non-virtual thunk to
QuantLib::FittedBondDiscountCurve::performCalculations() const'
collect2.exe: error: ld returned 1 exit status


Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd
--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com

Joshua Ulrich
 

On Sun, Dec 2, 2018 at 6:06 AM Joshua Ulrich <josh.m.ulrich@...> wrote:

On Sat, Dec 1, 2018 at 2:09 PM Dirk Eddelbuettel
<@eddelbuettel> wrote:

On 1 December 2018 at 14:06, Joshua Ulrich wrote:
| <snip>
| I'll follow up once the tests finish. I expect them to take at least
| an hour, maybe two.

Wicked. Thanks for setting that up.
Happy to. I want to find the source of this bug!

My time estimate was way off. Unit tests ran in ~20 minutes and
everything passed.

I'm currently trying to run the "windows_bug_redone.cpp" example, but
I'm having trouble linking. Here are the commands I'm running, and
the linking error.

g++ -g0 -O2 -std=c++11 -m64 -I./src/QuantLib-1.14 -I./src/boost_1_68_0 \
-L./QuantLib/lib/x64/lib -lQuantLib -c windows_bug_redone.cpp \
-o windows_bug_redone.o
g++ -static -o windows_bug_redone windows_bug_redone.o \
-L./QuantLib/lib/x64/lib -lQuantLib
<snip>
This error was an oversight on my part. I was including QuantLib
headers from ./src/QuantLib-1.14, *not* the installed headers at
./QuantLib/include.

The example builds, links, and runs once I make that change. So the
segfault with RQuantLib is likely a similar issue!

I'm currently waiting for the RQuantLib build to finish, so I can test
it with my newly-build QuantLib library.

--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com

Joshua Ulrich
 

On Sun, Dec 2, 2018 at 6:46 AM Joshua Ulrich <josh.m.ulrich@...> wrote:

On Sun, Dec 2, 2018 at 6:06 AM Joshua Ulrich <josh.m.ulrich@...> wrote:

On Sat, Dec 1, 2018 at 2:09 PM Dirk Eddelbuettel
<@eddelbuettel> wrote:

On 1 December 2018 at 14:06, Joshua Ulrich wrote:
| <snip>
| I'll follow up once the tests finish. I expect them to take at least
| an hour, maybe two.

Wicked. Thanks for setting that up.
Happy to. I want to find the source of this bug!

My time estimate was way off. Unit tests ran in ~20 minutes and
everything passed.

I'm currently trying to run the "windows_bug_redone.cpp" example, but
I'm having trouble linking. Here are the commands I'm running, and
the linking error.

g++ -g0 -O2 -std=c++11 -m64 -I./src/QuantLib-1.14 -I./src/boost_1_68_0 \
-L./QuantLib/lib/x64/lib -lQuantLib -c windows_bug_redone.cpp \
-o windows_bug_redone.o
g++ -static -o windows_bug_redone windows_bug_redone.o \
-L./QuantLib/lib/x64/lib -lQuantLib
<snip>
This error was an oversight on my part. I was including QuantLib
headers from ./src/QuantLib-1.14, *not* the installed headers at
./QuantLib/include.

The example builds, links, and runs once I make that change. So the
segfault with RQuantLib is likely a similar issue!

I'm currently waiting for the RQuantLib build to finish, so I can test
it with my newly-build QuantLib library.
RQuantLib built and I was able to successfully run the
ZeroCouponBond.R script without a segfault!

My conclusion is that something was incorrectly specified with my
build script. One thing I know I did not include was the `-std=c++11`
flag on the QuantLib build; that may have been the source of the
crashes.

I've only built everything for 64-bit Windows. I still need to create
the 32-bit builds. I will clean up my build script and re-run fresh,
to make sure I'm not missing any crucial details before I share with
the group.

--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com

Dirk Eddelbuettel
 

On 2 December 2018 at 09:09, Joshua Ulrich wrote:
| RQuantLib built and I was able to successfully run the
| ZeroCouponBond.R script without a segfault!

Nice!

| My conclusion is that something was incorrectly specified with my
| build script. One thing I know I did not include was the `-std=c++11`
| flag on the QuantLib build; that may have been the source of the
| crashes.

Quite possibly. This is really good news.

| I've only built everything for 64-bit Windows. I still need to create
| the 32-bit builds. I will clean up my build script and re-run fresh,
| to make sure I'm not missing any crucial details before I share with
| the group.

I am quite hopeful that we can restore the windows build to full capabilities
with that. Which really is pretty excellent news.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

shrektan
 

Josh,

One thing I know I did not include was the `-std=c++11`
| flag on the QuantLib build; that may have been the source of the
| crashes.

Great!

Yes, I believe so. With the flag `-std=gnu++11`, I build the QL 1.14 with Rtools35.
The windows_bug_redone.cpp file works fine linking to the new static library file.

Xianying

Dirk Eddelbuettel
 

Josh, Shrektan,

On 2 December 2018 at 21:01, shrektan wrote:
| Josh,
|
| | One thing I know I did not include was the `-std=c++11`
| | flag on the QuantLib build; that may have been the source of the
| | crashes.
|
| Great!
|
| Yes, I believe so. With the flag `-std=gnu++11`, I build the QL 1.14 with Rtools35.
| The windows_bug_redone.cpp file works fine linking to the new static library file.

Fantastic!

I will prepare a branch in the rquantlib repo in which I re-enable all the
Windows tests -- hopefully by this evening. We can then see how it fares.

So with some luck we may get an update libquantlib to github's rwinlib, and
then an updated RQuantLib to CRAN.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

Joshua Ulrich
 

On Mon, Dec 3, 2018 at 6:08 AM Dirk Eddelbuettel
<@eddelbuettel> wrote:
<snip>

I will prepare a branch in the rquantlib repo in which I re-enable all the
Windows tests -- hopefully by this evening. We can then see how it fares.
I have done that this morning. I reverted all the changes in man/*
and tests/* from commit ca06fe8ec39919fa9ac7d876d2299b86ceb2b3f1 (see
comment about the patch below).

I was able to successfully build and run `R CMD check` using my
updated libQuantLib.a (after modifying src/Makevars.win to point to my
local QuantLib and Boost headers and QuantLib library).

So with some luck we may get an update libquantlib to github's rwinlib, and
then an updated RQuantLib to CRAN.
To test our luck, I have put my updated QuantLib install, build
script, patch to revert the RQuantLib Windows-only changes, and the
RQuantLib Makevars.win on my website:
https://www.joshuaulrich.com/RQuantLib/

Whoever can, please download and test.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd



--
Joshua Ulrich | about.me/joshuaulrich
FOSS Trading | www.fosstrading.com
R/Finance 2018 | www.rinfinance.com

Morgan
 

It is working perfectly for me now. Here is what i did. I am using Rtools 3.5.0.4

To create an exe file :
Rtools\mingw_64\bin\g++.exe -std=gnu++11 -O2 -Wall -fopenp -Wno-deprecated-declarations -o windows_bug_redone windows_bug_redone.cpp -ID:\QuantLib\QuantLib\include -LD:\QuantLib\ -lQuantLib

Note:
-I used the boost from the github repo
-I copied libQuantLib.a x64 version to D:\QuantLib

Output i get:
NPV 83.7214
Clean Price 84.3136
Dirty Price 84.3136
Accrued coupon 0
Yield 0.0429401

On Mon, 3 Dec 2018 12:28 Joshua Ulrich <josh.m.ulrich@... wrote:
On Mon, Dec 3, 2018 at 6:08 AM Dirk Eddelbuettel
<dirk.eddelbuettel@...> wrote:
>
<snip>
>
> I will prepare a branch in the rquantlib repo in which I re-enable all the
> Windows tests -- hopefully by this evening. We can then see how it fares.
>
I have done that this morning.  I reverted all the changes in man/*
and tests/* from commit ca06fe8ec39919fa9ac7d876d2299b86ceb2b3f1 (see
comment about the patch below).

I was able to successfully build and run `R CMD check` using my
updated libQuantLib.a (after modifying src/Makevars.win to point to my
local QuantLib and Boost headers and QuantLib library).

> So with some luck we may get an update libquantlib to github's rwinlib, and
> then an updated RQuantLib to CRAN.
>
To test our luck, I have put my updated QuantLib install, build
script, patch to revert the RQuantLib Windows-only changes, and the
RQuantLib Makevars.win on my website:
https://www.joshuaulrich.com/RQuantLib/

Whoever can, please download and test.

> Dirk
>
> --
> http://dirk.eddelbuettel.com | @eddelbuettel | edd@...
>
>
>


--
Joshua Ulrich  |  about.me/joshuaulrich
FOSS Trading  |  www.fosstrading.com
R/Finance 2018 | www.rinfinance.com



Dirk Eddelbuettel
 

On 4 December 2018 at 12:42, Morgan wrote:
| It is working perfectly for me now. Here is what i did. I am using Rtools
| 3.5.0.4
|
| To create an exe file :
| Rtools\mingw_64\bin\g++.exe -std=gnu++11 -O2 -Wall -fopenp
| -Wno-deprecated-declarations -o windows_bug_redone windows_bug_redone.cpp
| -ID:\QuantLib\QuantLib\include -LD:\QuantLib\ -lQuantLib
|
| Note:
| -I used the boost from the github repo
| -I copied libQuantLib.a x64 version to D:\QuantLib
|
| Output i get:
| NPV 83.7214
| Clean Price 84.3136
| Dirty Price 84.3136
| Accrued coupon 0
| Yield 0.0429401

Fabulous. Thanks for running it. One minor typo: '-fopenmp' lacks the m above.

I pushed a branch to RQuantLib which does away with one set of the test
castration. There is still another in three help pages (indexed by i386) but
that is older / may persist.

Josh: Any news on a full i386 / x86_64 build of the library?

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

Joshua Ulrich
 

The zip file on my site includes libraries for both builds, as well as a patch that I believe reverts all the Windows specific changes you made.


On Tue, Dec 4, 2018, 7:30 AM Dirk Eddelbuettel <dirk.eddelbuettel@... wrote:

Josh: Any news on a full i386 / x86_64 build of the library?

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | edd@...

Dirk Eddelbuettel
 

On 4 December 2018 at 07:33, Joshua Ulrich wrote:
| The zip file on my site includes libraries for both builds, as well as a
| patch that I believe reverts all the Windows specific changes you made.

Cool, thanks.

Any plans to push this it github.com/rwinlib/quantlib ?

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

W. D.
 

everything runs like a charm 
took me ages to build Quantlib etc. but If needed I have .zip at hand now 
Or what would be the next expected step? 

G, Walter

W. D.
 

I put something here 
https://github.com/ViennaR/rquantlib 

G, W

Dirk Eddelbuettel
 

On 4 December 2018 at 06:31, W. D. wrote:
| I put something here 
| https://github.com/ViennaR/rquantlib 

Have a look at https://github.com/rwinlib -- this is a bit more formalized
for multiple projects providing their windows libraries there. We are just
one.

Once things work again maybe we can think about refreshing some of the code.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

W. D.
 

Hmm ... I do not have push access there ... otherwise I could refresh the lib.zip (with a bit of directory reshuffle before zipping it, my folder structure for lib is $ARCH/lib and not lib/$ARCH – guess this should not be a big issue)  

Otherwise I'll gladly help out in any way as much as I can ...

 

G, W

 

Sent from Mail for Windows 10

 

From: Dirk Eddelbuettel
Sent: 04 December 2018 15:55
To: rquantlib@groups.io
Subject: Re: [rquantlib] boost unit tests

 

 

On 4 December 2018 at 06:31, W. D. wrote:

| I put something here 

| https://github.com/ViennaR/rquantlib 

 

Have a look at https://github.com/rwinlib -- this is a bit more formalized

for multiple projects providing their windows libraries there.  We are just

one.

 

Once things work again maybe we can think about refreshing some of the code.

 

Dirk

 

--

http://dirk.eddelbuettel.com | @eddelbuettel | edd@...

 

 

 

Dirk Eddelbuettel
 

On 4 December 2018 at 17:22, W. D. wrote:
| Hmm ... I do not have push access there ...

That is normal. Neither do I. But Josh does, and he prepared the library we
tested here.

| otherwise I could refresh the lib.zip (with a bit of directory reshuffle
| before zipping it, my folder structure for lib is $ARCH/lib and not
| lib/$ARCH – guess this should not be a big issue) Otherwise I'll gladly
| help out in any way as much as I can ...

Testing helps, and you did that.

Folder structure does matter for deployment as the only way to automate this
across CRAN, win-builder, RHub, ... is to conform to the structure. The
posted library file lays one out, my RQuantLib package uses it. All good.

For local testing we can always adjust as you did.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd

shrektan
 

Hi,

Is it possible to automate the process in Appveyor, including building QuantLib with gcc?
According to the website, Appveyor's environment already installs MinGW and MSYS... 

I want to take a try but have no idea about the difficulties...

Xianying

Dirk Eddelbuettel
 

On 5 December 2018 at 06:48, shrektan wrote:
| Is it possible to automate the process in Appveyor, including building QuantLib with gcc?
| According to the website, Appveyor's environment already installs MinGW and MSYS... 

Yes and no.

Yes, in theory it would be possible.

But "No" because

- this is still complicated and requires a lot of hand-holding
- it is not just gcc but whatever EXACT config Rtools has
- do we really need it repeatedly ?

What we _might_ want to do consider is testing RQuantLib this way but I would
even call that a no because

- I had to work reasonably hard to accomodate Travis CI on Linux
- I now mostly 'avoid' it by relying more on Docker

| I want to take a try but have no idea about the difficulties...

The nice thing is that you can try now. Just fork and set up appveyor /
circle ci / travis ci / ... files to your heart's content. You would then
have to convince the "VMM" (== "very mean maintainer", ie me) to merge
whatever you send. But hey if you work some magic out we can talk.

I just don;t think it will be easy as this is a BIG package with a BIG
dependency.

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | @edd