THE SQL Server Blog Spot on the Web

Welcome to SQLblog.com - The SQL Server blog spot on the web Sign in | |
in Search

SQLBI - Marco Russo

SQLBI is a blog dedicated to building Business Intelligence solutions with SQL Server.
You can follow me on Twitter: @marcorus

  • Q&A from 24 hour of PASS #pass24hop #powerbi

    One week ago I delivered a session for 24 hour of PASS, the online free event delivered by PASS (recordings are now available), where I introduced my one-day preconference Create a Power BI Solution in one day that I will deliver at PASS Summit 2016 in Seattle on October 25, 2016.

    As usual, there were too many questions and not enough time, so I include in this blog post the Q&A that I was not able to answer online. I hope it will be helpful.

    • Can you give us some information about the best way to govern security for accessing reports?
      • This is a long topic that you can understand better by reading the free eBook Introducing Microsoft Power BI.
      • In short, you can share a dashboard from your personal workspace (you can invite people from outside your organization in this case), or you can create a group workspace within your organization so that all the members access to all documents without requiring single authorization for each dashboard. You also have organizational content packs as a way to deliver shared content within an organization.When we can get a solution of BI like Power BI without to have publish my data on the cloud?
    • Do your company's network administrators have to open ports in order for the gateway to work?
      • The Data Gateway is like a client connecting to web services through ports 80/443, plus a few other outbound ports that have to be opened. You can find a detail here in section Ports.
    • Can PowerBI connect to an on-premise SQL Server OLTP database or only to an Analysis Services database?
      • You can do both. The connection with SQL Server could be in Import or DirectQuery mode, the one with Analysis Services could be in Import or Live mode. Fundamentally, Import creates a copy of data on Power BI service that you can refresh, and data are available even if your gateway is not accessible. Using DirectQuery / Live connections, data are not stored in Power BI service, but your on-premise server must we available at query time.
    • When using the Data Gateway with a windows user where does the AD that authenticates that user can reside?
      • The data gateway connects to Analysis Services using an administrator, and it can impersonate an user using the EffectiveUserName property in the connection string.
      • I suggest you reading the Power BI Security article written by Adam Saxton.
    • Any thoughts about the row-level security introduced in the July release of Power BI versus the use of row-level security in SSAS?
      • The row-level security is fundamentally the same feature you have in Analysis Services, just exposed through Power BI.
  • BLANK and Boolean functions like IF in #dax

    A recent change in the DAX language transformed the behavior of IF statement, so that it should not return BLANK but only TRUE/FALSE if the results should be logical expressions. In that case, the BLANK is transformed in a FALSE condition.

    For example, consider that the result of the following expression is FALSE and not BLANK:

    IF ( 1 = 1, BLANK(), TRUE )

    I wrote a longer explanation of that, thanks to Jeffrey Wang who provided the details of the implementation. Also a big thanks to Darren Gosbell, who raised the initial question.

  • Upcoming conference speeches and workshops in 2016 #ssas #tabular #dax #powerpivot #powerbi

    The summer is almost over and while we are working on new content (books and other for www.sqlbi.com), I already have the plans for this Autumn’s conferences.

    If you are interested in attending the PASS Summit 2016, don’t miss 24 hours of PASS (live online, September 7-8, 2016), I will preview the full-day seminar about Power BI on 07 Sep 2016 21:00 GMT. This event is free, you just have to register, and there are many other interesting sessions to watch.

    I and Alberto Ferrari will also also have a number of public trainings:

    The course about Analysis Services Tabular Workshop is renewed and updated to Analysis Services 2016. The one in Amsterdam will be the first delivery in a public classroom, depending on the demand, we will propose new dates in 2017.

    See you around the world!

  • Update custom visuals on OKViz (and name survey result) #powerbi

    In the last few days, users of the Synoptic Panel and Smart Filter (custom visuals for Power BI) experienced some issue in the behavior of these components. Changes applied to API and automatic updates pushed through the Microsoft Power BI Gallery created some unexpected problems. Now the components are synchronized on www.okviz.com and the Power BI Gallery. If you have used the components in Power BI Desktop, make sure to download and update the components the latest version available, and if necessary publish the report on Power BI service, too.

    We worked to make sure such a disruption will not happen again! We also have some interesting improvements for Smart Filter, but we have to make sure certain API will stabilize before deploying them.

    In the meantime we closed the survey for the OKViz name, and the result is… stay with OKViz! Complete results are available here.

  • Leverage INTERSECT to apply relationships in DAX

    If you are used to virtual relationships in DAX (see Handling Different Granularities in DAX), you probably use the following pattern relatively often:

    [Filtered Measure] :=
    CALCULATE (
        <target_measure>,
        FILTER (
            ALL ( <target_granularity_column> ),
            CONTAINS (
                VALUES ( <lookup_granularity_column> )
                <lookup_granularity_column>,
                <target_granularity_column> 
            )
        )
    )

    In the new DAX available in Excel 2016*, Power BI Desktop, and Analysis Services 2016, you can use a simpler syntax, which offers a minimal performance improvement and is much more readable:

    [Filtered Measure] :=
    CALCULATE (
        <target_measure>,
        INTERSECT (
            ALL ( <target_granularity_column> ),
            VALUES ( <lookup_granularity_column> )
        )
    )

    You can find a longer explanation of this new pattern and download some examples in the new article Physical and Virtual Relationships in DAX, on SQLBI web site.

  • Happy Birthday Power BI #powerbi

    Power BI has been on the market one year. My biggest concern, when the product was still in private beta, was the promise of monthly releases made by Microsoft. Today, I can say that the promise was real. I see a long road ahead, in terms of features and possible improvements. But it’s a matter of fact that Power BI is a product with a growing user adoption, that every month adds features that increase the number of companies and users that can consider its usage.

    If you go back to the situation of Microsoft BI two years ago, you should remind the lack of a mobile story, the requirement for SharePoint and the poor situation on the client side. A great server product (such as Analysis Services) was limited in its adoption because of the client options available. Today the trend is completely different, and on certain areas of the BI platform Microsoft became the leader instead of a follower.

    I always try to find the missing part, what can be improved, without spending too much time praising what is good (and there are many things that are). But, today, I just want to join the choir you will see in the video:

    HAPPY BIRTHDAY POWER BI !

  • A new MemoryHeapType default in #ssas #tabular 2016 (please, fix your setting in production!)

    If you already installed Analysis Services 2016, you should change the MemoryHeapType setting. There is a new default value (-1), which is an automatic choice that currently applies a new hybrid allocator (which has the number 5 as a value). It should resolve the memory fragmentation problem causing performance issue as described in an article I wrote a few years ago. However, the setup does not write the new default value as a current value and it still write the old default “2”, which is not good for Tabular. Thus, if you installed SSAS Tabular 2016, you probably have this setting (look at the Default Value different than the Value!). The new setting is also the suggested one for Multidimensional.

    image

    You should change the value to -1 and then restart the service. After that, reconnect to SSAS Properties and check that you have the following configuration:

    image

    Of course, we hope future updates of SQL 2016 will fix this setup issue. In the meantime, fix the setting to avoid performance issues on a production server!

  • Free Introducing Power BI eBook and new DAX recorded video course #powerbi #dax

    Microsoft Press released a free eBook you can already download, Introducing Microsoft Power BI, which I and Alberto Ferrari wrote in the last few months. Please note it is a very introductive book, don’t expect an inside-out. As we wrote in the introduction:

    analyticsWe wanted to write an introduction to Power BI that covers the basics of the tool and, at the same time, shows you what the main capabilities of Power BI are. […] At the beginning, we go for an easy introduction of the concepts along with an educational approach that lets you follow on your PC the same steps we show in the book. […] After the first chapters, we begin to run a bit faster, knowing that we are no longer guiding you step by step. […]

    This book is targeted to a variety of readers. There are information workers and people who are totally new to the BI world. For those readers, the book acts as a simple introduction to the concepts that are the foundation of BI. Yet, another category of we wanted to target is that of IT professionals and database administrators who might need to drive the decisions of the company in adopting Power BI, because their users are asking for it. If this is you, this book acts as both a simple introduction to the basic concepts, to help you understand why users are so interested in Power BI, and as an overview of the capabilities and tools available in Power BI, so that you can make educated choices in adopting it.

    As you see in the side picture, we included some real-world reports, and we have an entire chapter titled “Improving Power BI reports” where you will find a number of useful examples and guidelines. And, of course, they are included in the companion content, which is a separate download available here. We used some of the components available ok okviz.com, even if when we wrote the book such a web site was not ready, so we didn’t mention it in the book.

    masteringdaxThe goal of the book is to be introductive. So, what’s next? We are working on some new content for later this year, but in the last few months we also worked on a recorded version of our Mastering DAX course. So, if you cannot join us in one of the many classes we deliver around the world, you can now get a recorded version, which is complete with all the exercises. You will not have the same interaction that is possible in a classroom, but early adopters who also attended the live class told us that getting a recorded video as a revise tool. The structure and the flow is the same, and we tried to compensate the lack of interactivity with a physical presence on the screen.

    You can watch a number of segments for free, and you can save 70 USD until July 5 getting access to the entire course.

  • Many-to-many relationships in DAX and CROSSFILTER #dax #powerbi #powerpivot #ssas #tabular

    I wrote many articles and white papers about implementing many-to-many relationships in Analysis Services - for both Multidimensional and Tabular models. For the latter, the same techniques are also useful in Power Pivot and Power BI. With the recent release of Excel 2016, Analysis Services 2016, and the engine available in Power BI Desktop, there are new tools and techniques that we can use to implement this type of data models, which are more efficient and easier to manage in your DAX code.

    The article Many-to-many relationships in Power BI and Excel 2016 describes how to use the bidirectional filters in Power BI and Analysis Services 2016, and also how to obtain the same performance in Excel 2016 by using the CROSSFILTER function. These techniques use a smart technique that applies the filter through the relationships in the many-to-one direction only when this is really necessary, because there is an existing filter over the bridge table in a many-to-many relationship. This is possible in the "old" DAX with a more complex conditional statement, which makes the code less readable and also suffer of another performance issue related to the usage of an IF statement in a measure (also solved in the "new" engine).

    I think there are a huge number of data modeling options that are now possible thanks to these innovations, many of them will have an impact on several DAX Patterns, which we will revisit in a few months using the new techniques. 

  • Licensing and upgrade options for Analysis Services Tabular in SQL Server 2016 #ssas #tabular

    With the release of Microsoft SQL Server 2016, there are several options to use Analysis Services Tabular.

    • SQL Server Standard: it includes Analysis Services Tabular, and it is available in two licensing models: Server + Client Access License (CAL) and Core-based.
    • SQL Server Enterprise: it has all the features, and is available only in a license per-core.

    For a Tabular model, the limitations existing for the Standard edition affect the memory available for an instance (16GB) and the lack of these features, available only in Enterprise: DirectQuery, partitions, and perspectives. The limit of 24 cores for the standard fundamentally is a non-issue, because with the right hardware with a single socket you should have so many cores (no NUMA is still better for Tabular, at least in current release of 2016). You can find more details in this detailed matrix of the features available by the Editions of SQL Server 2016.

    If you have a Tabular model in SSAS 2012 or SSAS 2014 today, you might want to upgrade immediately because of the big performance improvements available in SSAS 2016 (almost any data model and report should benefit). However, what are the options to upgrade? It depends on your current license and whether you have a software assurance (SA) in place or not.

    • SQL 2012/14 Enterprise with SA: you can upgrade today, you should be already covered.
    • SQL 2012/14 Business Intelligence with SA: for the duration of your current SA contract, you can upgrade to SSAS 2016 using a licensing model Server+CAL that is not available otherwise. At the end of your SA, you will have to renew using current SQL Server Enterprise licensing terms. You can also get new licenses (of Business Intelligence that you upgrade in this way) up to 25% of the number you had on May 1, 2016.
    • SQL 2012/14 Enterprise or Business Intelligence without SA: you have to buy a license for SQL 2016. You might consider using the Standard for SSAS Tabular in case your model does not need the features available in Enterprise, otherwise you should get an Enterprise license.

    Please note this is a summary I created to recap the current situation. I suggest you to consider all details and terms explained in this PDF. You can also visit this Microsoft website with all the information about SQL Server licensing.

    Finally, a very welcome news is that Microsoft SQL Server 2016 Developer edition is now free. It had a minimal cost in the past, but now it will be much easier to install development enviroments also in those companies with a very long and bureaucratic procurement process, other than for all the consultants who want to study the new environment on their workstations without using a trial version expiring in 6 months.

  • New formula to compute new customers in #dax

    Two years ago I and Alberto Ferrari spent time discussing techniques to calculate new and returning customers, and we wrote the result of our study in the New and Returning Customer article in DAX Patterns web site and book.

    During one of the Mastering DAX courses, a student presented a better solution and Alberto published an article describing this approach that is much faster than the ones we used before. We also worked to backport this algorithm in the "legacy" DAX, without using new features available only in Power BI Desktop, Excel 2016, and SSAS 2016 (we call it "DAX 2015" to quickly identify the DAX version). The performances are equally amazing and you can read the full story (and the DAX code) on the Computing New Customers in DAX article on SQLBI.

    In the future we will also update the DAX Patterns web site, but the idea is to wait a larger work on the existing patterns updating them using the new syntax available in DAX 2015.

  • Leverage Analyze in Excel for your Power Pivot model

    Microsoft recently released the Analyze in Excel feature, which allows you to connect an Excel pivot table to a data model published on Power BI. I think/hope that Microsoft will enhance this feature to extend its current limits. For example, it does not allow you to connect a live model based on an on-premises SSAS Tabular server through Power BI Enterprise gateway. However, this feature is also very interesting to any Power Pivot user.

    When you have a data model in Power Pivot, you have to create all the reports within the same Excel workbook. At the end, you have a single file containing everything: the data model, the data, and the reports. This is fine at the beginning, but as soon as you create more complex data models, you increase the number of reports – in a word, when the model grows – then you face a few problems. Any edit operation of a power pivot model could require time to refresh pivot tables and measures. The editing experience becomes slower. You cannot separate the data from the reports in two different files to split maintenance responsibility. If you created several reports based on the same power pivot data model, you already know this story. It would be nice to split the data model (and the data) from the reports. Now you can, using Power BI and the Analyze in Excel feature. Power BI is your best friend here.

    This is the list of the operations you have to do:

    1. Download and install Power BI Desktop – you have to be a local administrator of your machine to do that, but this is the only step requiring such a permission.
    2. Open Power BI Desktop.
    3. Import the Power Pivot data model in Power BI Desktop using the feature File / Import / Excel Workbook Contents, and then schedule a refresh if you need that.
      import-from-excel
    4. Publish the Power BI Desktop data model in Power BI service
    5. Open the data model from Excel using the Analyze In Excel feature on Power BI service
    6. Create the same reports you had in the original Excel file using new pivot tables.

    At the end, you will have an Excel file connected to an external data model, which can refresh automatically through a schedule you control. Before Power BI, you had to use Power Pivot for SharePoint to do that.

    You might ask why I am not suggesting to publish the Power Pivot data model straight to Power BI service, without creating a Power BI Desktop file to do that. The reason is that it is much simpler to work with two separate files, and the environment in Power BI Desktop provides you the ability to use bidirectional filter in relationships, a feature that is not available in Power Pivot for Excel. However, this approach requires you to plan in advance the moment you “detach” the data model from Excel if you defined synonyms for Q&A.

    At the moment of writing, you cannot modify synonyms in Power BI, neither in Power BI desktop nor in Power BI service. However, if you created synonyms in Excel, they are kept in the data model in Power BI Desktop, even if you cannot modify them (by now, I hope!). If you want to modify the synonyms for Q&A in the Excel file, you have to repeat the cycle and import the Power Pivot data model again in Power BI, losing the changes you might have done before. I am not saying that this way of working should be suggested to anyone, but you might be interested if you are willing to pay the maintenance cost of this solution, which already provides you the best of the two worlds: define synonyms for Q&A in Power Pivot, and create relationships with bidirectional filter in Power BI Desktop. When editing synonyms will be possible also in Power BI Desktop and/or in Power BI Service, this article will become immediately obsolete.

    At the moment, you should consider that changing the synonyms in Power Pivot will require you to overwrite the changes you applied to the data model in Power BI Desktop. Thus, you might want to apply any change to the data model in Power Pivot (for example, adding/removing columns/measures) instead of using Power BI. You might just apply bidirectional filter on Power BI, but you will have to repeat that every time you import the data model in Power BI Desktop from Excel again. And you would lose any report designed in Power BI Desktop doing that, so you should edit reports in Power BI service only… Editing synonyms in Power BI will definitely streamline the process, when it will be possible.

    One feature that I would like for this scenario is being able to change automatically all the connections of the pivot tables based on a Power Pivot data model, replacing the correspondent connection string in the ODC file returned by Power BI clicking on Analyze In Excel. This is also a long awaited feature for those who want to publish a Power Pivot data model to an on-premises Analysis Services Tabular server, and it is not easy to manipulate the file so that you can refactor the connection string in all the queries of the dataset. I am sure that this would improve the productivity of Excel users facing this issue, and it will increase the number of users or Power BI service.

    Conclusion

    You can use synonyms today in Q&A, but you have to create and edit the model in Power Pivot, exporting it to Power BI only to edit relationships using types not available in Power Pivot (such as the bidirectional filter).

  • A warm welcome to Power BI Embedded #powerbi

    I wrote this blog post with Alberto Ferrari.

    A few days ago, during the Build conference, Microsoft announced the availability, although still in preview, of Power BI Embedded. In a few words, Power BI Embedded lets you embed Power BI reports in your application, taking advantage of the tools in Power BI to enrich the analytical power of your application. Users do not have to authenticate with the Power BI service. In fact, they do not even need a Power BI account. Your application performs the necessary steps to authenticate a user and then, it uses App Tokens to request for report rendering. App Tokens are generated by the Power BI service when your application requests them, providing the necessary keys associated with your Azure subscription.

    Power BI embedded is priced using renders. Currently, the price is 2.50 USD per 1,000 renders, and the first 1,000 renders in a month are free. A render is the production of a visual in a report, so a single report requires one or more renders (a dense report will be more expensive).

    The first step to use Power BI Embedded is to activate the service in Azure, by means of creating a Power BI Workspace Collection. A workspace collection, as its name implies, is a set of workspaces. A workspace can contain datasets, reports and dashboards, same as it is happening today with the Power BI workspaces you work with when you are connected to the Power BI service.

    You can add items to a Power BI Embedded workspace by uploading a PBIX file generated with Power BI Desktop. Connection strings, data model, measures and calculated columns, visual interactions and all other functionalities of Power BI are uploaded along with the model and available for your reports. This is, at least, the experience during the preview. As soon as the product reaches general availability, the development experience might be different and, as of today, there are no information about how it will work when released (hopefully you will be able to automate the creation of the model and its data refresh). Nevertheless, in this early phase, the option of using Power BI Desktop for development is awesome, as it leverages a well-established technology to author the reports, although it means that – as of today – you cannot add a dashboard to a workspace collection, because Power BI Desktop does not have a dashboard authoring feature, it stops with reports. Another limitation that exists during the preview is that you cannot refresh data, unless you upload a new version of the PBIX file. If you need to work with live data, today the only option is to leverage DirectQuery connections, along with all the limitations that come from that.

    Once the workspace collection is in place, your application needs to connect to the service and generate an App Token by providing the needed key (which you have to store in the application itself). Once the application has an App Token, it can interact with the user the way it needs and, when it is time to produce a dashboard, it provides the App Token to the service again requesting the rendering of one or more reports. Power BI takes care of handling the report rendering and visual interactions.

    In order for the entire reporting system to work, the Power BI service needs to be able to connect to the data source. This can be very easy, if the data source is already in the cloud, or it requires some additional refreshing steps if the data source is on premises and not connected to the service. In such a case, you have to publish to the Power BI cloud service an updated version of the PBIX file using the Power BI Embedded API.

    So far, so good, it looks like Microsoft created yet another library to show dashboards inside an app. Why do we believe this is huge, not just yet another reporting app?

    • It uses the very same tools your users are probably using in Power BI to gather insights.
    • It does not have a starting price: you pay for what you use. The more customers are using your application, the more you pay for. By using a similar approach (pay per use) you can limit the initial investment to build a real analytical system in your application
    • It does not provide you a simple reporting tool. It provides a modeling tool where you build a data model through a Power BI Desktop file, along with measures, ETL steps with Power Query, calculated columns and tables. Thus, in order to provide analytics to your application, you do not need to focus on how to build a given report. Instead, you will focus on how to build an analytical model on top of which, later, you will build reports.

    This latter change is, in our opinion, the most important of all. A simple shift in the way you think at reporting inside an application might open a whole bunch of new features and help democratizing Business Intelligence. For example, it is easy to think at standard reporting provided inside the application and, for advanced users that require more power, ISV can provide Power BI Desktop files that can be customized and later deployed on Power BI to perform custom analytics.

    It is important to note that this preview is not a complete solution for any ISV. Today there are no APIs to programmatically create a PBIX file. This results in strong limitations for automating the creation of a custom data model depending on parameters defined by the application created by the ISV. An API to create a PBIX file or to manipulate a data model published on the server would be an important advancement to create a fully functional ecosystem for the ISVs. We hope this is only the first step in that direction.

  • How to fix #dax time intelligence functions in #powerbi when date table uses a surrogate key

    I wrote an article about Time Intelligence in Power BI Desktop to explain how to fix measures using DAX functions for time intelligence when you have a data model where the relationship uses surrogate keys. THe short description is that all the time intelligence functions in DAX makes two assumptions: you have a "full" date table (all days for each year you want to handle must be present), and you use the "Mark as Date Table" setting to identify the date column in a date table. However, the latter is not possible in Power BI Desktop, because such a setting is not available yet (end of February 2016).

    There are a number of workarounds possible when we wait for such a feature in a future version of Power BI Desktop. This problem has a limited impact because, when you use the date column in the relationship, the "Mark as Date Table" setting is not strictly necessary. However, it is a good idea to understand why this behavior exists and how the settings affect the DAX engine by reading the article, even if you don't have this problem today.

  • Possible #powerpivot issue removing Excel 2016 going back to Excel 2013

    Today I observed an issue on a notebook where Excel 2016 was installed and then removed, going back to Excel 2013.

    After this operation, workbooks having a Power Pivot data model were working correctly by navigating data in PivotTables, but any operation in the Power Pivot window (such as adding columns, tables, or editing measures) was failing. The error was a TYPE_E_CANTLOADLIBRARY and it was caused by a type library not removed uninstalling Excel 2016.

    I solved the problem by removing the following key from the registry 

    [HKEY_CLASSES_ROOT\TypeLib\{00020813-0000-0000-C000-000000000046}\1.9]

    (WARNING: DON'T TOUCH THE WINDOWS REGISTRY IF YOU DON'T KNOW WHAT YOU ARE DOING - use the following link to get more detailed instructions and always do a backup first).

    I found this article that describes the problem as a possible issue in all the versions of Excel!

More Posts Next page »

This Blog

Syndication

Archives

Privacy Statement