illumos/git/illumos-gate: usr/src/uts/i86pc/io/rootnex.c annotate

annotate usr/src/uts/i86pc/io/rootnex.c @ 19431:5599be23ae32

12183 Want new IPD 13 DMA Cookie APIs Reviewed by: Alex Wilson <alex@uq.edu.au> Reviewed by: Paul Winder <paul@winders.demon.co.uk> Reviewed by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org> Reviewed by: Garrett D'Amore <garrett@damore.org> Approved by: Garrett D'Amore <garrett@damore.org>

author	Robert Mustacchi <rm@fingolfin.org>
date	Wed, 18 Dec 2019 06:18:35 +0000
parents	42631d95c231
children	063a4bde1fa8

rev	line source
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	2 * CDDL HEADER START
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	3 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	4 * The contents of this file are subject to the terms of the
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5 * Common Development and Distribution License (the "License").
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	6 * You may not use this file except in compliance with the License.
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	7 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	8 * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	9 * or http://www.opensolaris.org/os/licensing.
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	10 * See the License for the specific language governing permissions
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	11 * and limitations under the License.
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	12 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	13 * When distributing Covered Code, include this CDDL HEADER in each
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	14 * file and include the License file at usr/src/OPENSOLARIS.LICENSE.
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	15 * If applicable, add the following below this CDDL HEADER, with the
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	16 * fields enclosed by brackets "[]" replaced with your own identifying
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	17 * information: Portions Copyright [yyyy] [name of copyright owner]
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	18 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	19 * CDDL HEADER END
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	20 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	21 /*
12118 6928a154b915 6924464 fstyp on kimchee device panic the system Mark Johnson <mark.r.johnson@oracle.com> parents: 12027 diff changeset	22 * Copyright (c) 1992, 2010, Oracle and/or its affiliates. All rights reserved.
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	23 */
13268 b749961aba64 612 apparent reversal of dvma vs. dma in rootnex Garrett D'Amore <garrett@nexenta.com> parents: 13216 diff changeset	24 /*
b749961aba64 612 apparent reversal of dvma vs. dma in rootnex Garrett D'Amore <garrett@nexenta.com> parents: 13216 diff changeset	25 * Copyright 2011 Nexenta Systems, Inc. All rights reserved.
13452 6bec9720e054 1073 migrate kernel modules from ancient _depends_on to true ELF dependencies Bayard Bell <buffer.g.overflow@gmail.com> parents: 13364 diff changeset	26 * Copyright (c) 2011 Bayard G. Bell. All rights reserved.
13636 776c3d9cae69 998 obsolete DMA driver interfaces should be removed Garrett D'Amore <garrett@damore.org> parents: 13631 diff changeset	27 * Copyright 2012 Garrett D'Amore <garrett@damore.org>. All rights reserved.
16725 1a38f0e5c9d4 8624 xhci and nvme can't bind DMA memory with IOMMU enabled Hans Rosenfeld <hans.rosenfeld@joyent.com> parents: 15433 diff changeset	28 * Copyright 2017 Joyent, Inc.
13268 b749961aba64 612 apparent reversal of dvma vs. dma in rootnex Garrett D'Amore <garrett@nexenta.com> parents: 13216 diff changeset	29 */
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	30
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	31 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	32 * x86 root nexus driver
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	33 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	34
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	35 #include <sys/sysmacros.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	36 #include <sys/conf.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	37 #include <sys/autoconf.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	38 #include <sys/sysmacros.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	39 #include <sys/debug.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	40 #include <sys/psw.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	41 #include <sys/ddidmareq.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	42 #include <sys/promif.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	43 #include <sys/devops.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	44 #include <sys/kmem.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	45 #include <sys/cmn_err.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	46 #include <vm/seg.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	47 #include <vm/seg_kmem.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	48 #include <vm/seg_dev.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	49 #include <sys/vmem.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	50 #include <sys/mman.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	51 #include <vm/hat.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	52 #include <vm/as.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	53 #include <vm/page.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	54 #include <sys/avintr.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	55 #include <sys/errno.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	56 #include <sys/modctl.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	57 #include <sys/ddi_impldefs.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	58 #include <sys/sunddi.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	59 #include <sys/sunndi.h>
916 ed14fe603852 PSARC/2005/584 PSM_INTR_OPS extensions schwartz parents: 693 diff changeset	60 #include <sys/mach_intr.h>
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	61 #include <sys/psm.h>
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	62 #include <sys/ontrap.h>
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	63 #include <sys/atomic.h>
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	64 #include <sys/sdt.h>
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	65 #include <sys/rootnex.h>
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	66 #include <vm/hat_i86.h>
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	67 #include <sys/ddifm.h>
5251 aad4f9e16063 PSARC/2007/589 dma-max-arch scsi capability mrj parents: 5084 diff changeset	68 #include <sys/ddi_isa.h>
12683 92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	69 #include <sys/apic.h>
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	70
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	71 #ifdef __xpv
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	72 #include <sys/bootinfo.h>
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	73 #include <sys/hypervisor.h>
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	74 #include <sys/bootconf.h>
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	75 #include <vm/kboot_mmu.h>
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	76 #endif
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	77
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	78 #if defined(__amd64) && !defined(__xpv)
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	79 #include <sys/immu.h>
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	80 #endif
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	81
7589 7de800909a06 PSARC 2008/560 Intel IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7542 diff changeset	82
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	83 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	84 * enable/disable extra checking of function parameters. Useful for debugging
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	85 * drivers.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	86 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	87 #ifdef DEBUG
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	88 int rootnex_alloc_check_parms = 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	89 int rootnex_bind_check_parms = 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	90 int rootnex_bind_check_inuse = 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	91 int rootnex_unbind_verify_buffer = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	92 int rootnex_sync_check_parms = 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	93 #else
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	94 int rootnex_alloc_check_parms = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	95 int rootnex_bind_check_parms = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	96 int rootnex_bind_check_inuse = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	97 int rootnex_unbind_verify_buffer = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	98 int rootnex_sync_check_parms = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	99 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	100
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	101 boolean_t rootnex_dmar_not_setup;
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	102
1414 b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	103 /* Master Abort and Target Abort panic flag */
b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	104 int rootnex_fm_ma_ta_panic_flag = 0;
b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	105
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	106 /* Semi-temporary patchables to phase in bug fixes, test drivers, etc. */
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	107 int rootnex_bind_fail = 1;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	108 int rootnex_bind_warn = 1;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	109 uint8_t *rootnex_warn_list;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	110 /* bitmasks for rootnex_warn_list. Up to 8 different warnings with uint8_t */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	111 #define ROOTNEX_BIND_WARNING (0x1 << 0)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	112
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	113 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	114 * revert back to old broken behavior of always sync'ing entire copy buffer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	115 * This is useful if be have a buggy driver which doesn't correctly pass in
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	116 * the offset and size into ddi_dma_sync().
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	117 */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	118 int rootnex_sync_ignore_params = 0;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	119
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	120 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	121 * For the 64-bit kernel, pre-alloc enough cookies for a 256K buffer plus 1
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	122 * page for alignment. For the 32-bit kernel, pre-alloc enough cookies for a
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	123 * 64K buffer plus 1 page for alignment (we have less kernel space in a 32-bit
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	124 * kernel). Allocate enough windows to handle a 256K buffer w/ at least 65
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	125 * sgllen DMA engine, and enough copybuf buffer state pages to handle 2 pages
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	126 * (< 8K). We will still need to allocate the copy buffer during bind though
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	127 * (if we need one). These can only be modified in /etc/system before rootnex
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	128 * attach.
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	129 */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	130 #if defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	131 int rootnex_prealloc_cookies = 65;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	132 int rootnex_prealloc_windows = 4;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	133 int rootnex_prealloc_copybuf = 2;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	134 #else
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	135 int rootnex_prealloc_cookies = 33;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	136 int rootnex_prealloc_windows = 4;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	137 int rootnex_prealloc_copybuf = 2;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	138 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	139
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	140 /* driver global state */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	141 static rootnex_state_t *rootnex_state;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	142
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	143 #ifdef DEBUG
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	144 /* shortcut to rootnex counters */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	145 static uint64_t *rootnex_cnt;
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	146 #endif
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	147
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	148 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	149 * XXX - does x86 even need these or are they left over from the SPARC days?
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	150 */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	151 /* statically defined integer/boolean properties for the root node */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	152 static rootnex_intprop_t rootnex_intprp[] = {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	153 { "PAGESIZE", PAGESIZE },
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	154 { "MMU_PAGESIZE", MMU_PAGESIZE },
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	155 { "MMU_PAGEOFFSET", MMU_PAGEOFFSET },
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	156 { DDI_RELATIVE_ADDRESSING, 1 },
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	157 };
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	158 #define NROOT_INTPROPS (sizeof (rootnex_intprp) / sizeof (rootnex_intprop_t))
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	159
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	160 /*
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	161 * If we're dom0, we're using a real device so we need to load
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	162 * the cookies with MFNs instead of PFNs.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	163 */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	164 #ifdef __xpv
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	165 typedef maddr_t rootnex_addr_t;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	166 #define ROOTNEX_PADDR_TO_RBASE(pa) \
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	167 (DOMAIN_IS_INITDOMAIN(xen_info) ? pa_to_ma(pa) : (pa))
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	168 #else
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	169 typedef paddr_t rootnex_addr_t;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	170 #define ROOTNEX_PADDR_TO_RBASE(pa) (pa)
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	171 #endif
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	172
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	173 static struct cb_ops rootnex_cb_ops = {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	174 nodev, /* open */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	175 nodev, /* close */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	176 nodev, /* strategy */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	177 nodev, /* print */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	178 nodev, /* dump */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	179 nodev, /* read */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	180 nodev, /* write */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	181 nodev, /* ioctl */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	182 nodev, /* devmap */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	183 nodev, /* mmap */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	184 nodev, /* segmap */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	185 nochpoll, /* chpoll */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	186 ddi_prop_op, /* cb_prop_op */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	187 NULL, /* struct streamtab */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	188 D_NEW \| D_MP \| D_HOTPLUG, /* compatibility flags */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	189 CB_REV, /* Rev */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	190 nodev, /* cb_aread */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	191 nodev /* cb_awrite */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	192 };
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	193
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	194 static int rootnex_map(dev_info_t dip, dev_info_t rdip, ddi_map_req_t *mp,
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	195 off_t offset, off_t len, caddr_t *vaddrp);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	196 static int rootnex_map_fault(dev_info_t dip, dev_info_t rdip,
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	197 struct hat hat, struct seg seg, caddr_t addr,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	198 struct devpage *dp, pfn_t pfn, uint_t prot, uint_t lock);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	199 static int rootnex_dma_allochdl(dev_info_t dip, dev_info_t rdip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	200 ddi_dma_attr_t attr, int (waitfp)(caddr_t), caddr_t arg,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	201 ddi_dma_handle_t *handlep);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	202 static int rootnex_dma_freehdl(dev_info_t dip, dev_info_t rdip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	203 ddi_dma_handle_t handle);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	204 static int rootnex_dma_bindhdl(dev_info_t dip, dev_info_t rdip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	205 ddi_dma_handle_t handle, struct ddi_dma_req *dmareq,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	206 ddi_dma_cookie_t cookiep, uint_t ccountp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	207 static int rootnex_dma_unbindhdl(dev_info_t dip, dev_info_t rdip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	208 ddi_dma_handle_t handle);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	209 static int rootnex_dma_sync(dev_info_t dip, dev_info_t rdip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	210 ddi_dma_handle_t handle, off_t off, size_t len, uint_t cache_flags);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	211 static int rootnex_dma_win(dev_info_t dip, dev_info_t rdip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	212 ddi_dma_handle_t handle, uint_t win, off_t offp, size_t lenp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	213 ddi_dma_cookie_t cookiep, uint_t ccountp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	214 static int rootnex_dma_mctl(dev_info_t dip, dev_info_t rdip,
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	215 ddi_dma_handle_t handle, enum ddi_dma_ctlops request,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	216 off_t offp, size_t lenp, caddr_t *objp, uint_t cache_flags);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	217 static int rootnex_ctlops(dev_info_t dip, dev_info_t rdip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	218 ddi_ctl_enum_t ctlop, void arg, void result);
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	219 static int rootnex_fm_init(dev_info_t dip, dev_info_t tdip, int tcap,
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	220 ddi_iblock_cookie_t *ibc);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	221 static int rootnex_intr_ops(dev_info_t pdip, dev_info_t rdip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	222 ddi_intr_op_t intr_op, ddi_intr_handle_impl_t hdlp, void result);
12683 92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	223 static int rootnex_alloc_intr_fixed(dev_info_t , ddi_intr_handle_impl_t ,
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	224 void *);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	225 static int rootnex_free_intr_fixed(dev_info_t , ddi_intr_handle_impl_t );
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	226
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	227 static int rootnex_coredma_allochdl(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	228 ddi_dma_attr_t attr, int (waitfp)(caddr_t), caddr_t arg,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	229 ddi_dma_handle_t *handlep);
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	230 static int rootnex_coredma_freehdl(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	231 ddi_dma_handle_t handle);
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	232 static int rootnex_coredma_bindhdl(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	233 ddi_dma_handle_t handle, struct ddi_dma_req *dmareq,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	234 ddi_dma_cookie_t cookiep, uint_t ccountp);
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	235 static int rootnex_coredma_unbindhdl(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	236 ddi_dma_handle_t handle);
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	237 #if defined(__amd64) && !defined(__xpv)
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	238 static void rootnex_coredma_reset_cookies(dev_info_t *dip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	239 ddi_dma_handle_t handle);
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	240 static int rootnex_coredma_get_cookies(dev_info_t *dip, ddi_dma_handle_t handle,
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	241 ddi_dma_cookie_t *cookiepp, uint_t ccountp);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	242 static int rootnex_coredma_set_cookies(dev_info_t *dip, ddi_dma_handle_t handle,
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	243 ddi_dma_cookie_t *cookiep, uint_t ccount);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	244 static int rootnex_coredma_clear_cookies(dev_info_t *dip,
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	245 ddi_dma_handle_t handle);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	246 static int rootnex_coredma_get_sleep_flags(ddi_dma_handle_t handle);
7617 1e2a36b96537 6748924 fix lint warnings in putback for 6747701 Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7613 diff changeset	247 #endif
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	248 static int rootnex_coredma_sync(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	249 ddi_dma_handle_t handle, off_t off, size_t len, uint_t cache_flags);
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	250 static int rootnex_coredma_win(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	251 ddi_dma_handle_t handle, uint_t win, off_t offp, size_t lenp,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	252 ddi_dma_cookie_t cookiep, uint_t ccountp);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	253
13065 9377d65d657e 6955196 Intel IOMMU code should use higher-level abstraction interface (fix lint) Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 13062 diff changeset	254 #if defined(__amd64) && !defined(__xpv)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	255 static int rootnex_coredma_hdl_setprivate(dev_info_t dip, dev_info_t rdip,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	256 ddi_dma_handle_t handle, void *v);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	257 static void rootnex_coredma_hdl_getprivate(dev_info_t dip, dev_info_t *rdip,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	258 ddi_dma_handle_t handle);
13065 9377d65d657e 6955196 Intel IOMMU code should use higher-level abstraction interface (fix lint) Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 13062 diff changeset	259 #endif
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	260
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	261
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	262 static struct bus_ops rootnex_bus_ops = {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	263 BUSO_REV,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	264 rootnex_map,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	265 NULL,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	266 NULL,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	267 NULL,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	268 rootnex_map_fault,
13636 776c3d9cae69 998 obsolete DMA driver interfaces should be removed Garrett D'Amore <garrett@damore.org> parents: 13631 diff changeset	269 0,
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	270 rootnex_dma_allochdl,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	271 rootnex_dma_freehdl,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	272 rootnex_dma_bindhdl,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	273 rootnex_dma_unbindhdl,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	274 rootnex_dma_sync,
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	275 rootnex_dma_win,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	276 rootnex_dma_mctl,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	277 rootnex_ctlops,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	278 ddi_bus_prop_op,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	279 i_ddi_rootnex_get_eventcookie,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	280 i_ddi_rootnex_add_eventcall,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	281 i_ddi_rootnex_remove_eventcall,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	282 i_ddi_rootnex_post_event,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	283 0, /* bus_intr_ctl */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	284 0, /* bus_config */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	285 0, /* bus_unconfig */
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	286 rootnex_fm_init, /* bus_fm_init */
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	287 NULL, /* bus_fm_fini */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	288 NULL, /* bus_fm_access_enter */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	289 NULL, /* bus_fm_access_exit */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	290 NULL, /* bus_powr */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	291 rootnex_intr_ops /* bus_intr_op */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	292 };
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	293
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	294 static int rootnex_attach(dev_info_t *dip, ddi_attach_cmd_t cmd);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	295 static int rootnex_detach(dev_info_t *dip, ddi_detach_cmd_t cmd);
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	296 static int rootnex_quiesce(dev_info_t *dip);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	297
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	298 static struct dev_ops rootnex_ops = {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	299 DEVO_REV,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	300 0,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	301 ddi_no_info,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	302 nulldev,
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	303 nulldev,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	304 rootnex_attach,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	305 rootnex_detach,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	306 nulldev,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	307 &rootnex_cb_ops,
7656 2621e50fdf4a PSARC 2008/382 Fast Reboot Sherry Moore <Sherry.Moore@Sun.COM> parents: 7617 diff changeset	308 &rootnex_bus_ops,
2621e50fdf4a PSARC 2008/382 Fast Reboot Sherry Moore <Sherry.Moore@Sun.COM> parents: 7617 diff changeset	309 NULL,
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	310 rootnex_quiesce, /* quiesce */
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	311 };
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	312
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	313 static struct modldrv rootnex_modldrv = {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	314 &mod_driverops,
7542 42d86bdbde36 6560816 x86 drivers should not use SCCS keywords in user-visible strings Richard Bean <Richard.Bean@Sun.COM> parents: 7173 diff changeset	315 "i86pc root nexus",
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	316 &rootnex_ops
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	317 };
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	318
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	319 static struct modlinkage rootnex_modlinkage = {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	320 MODREV_1,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	321 (void *)&rootnex_modldrv,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	322 NULL
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	323 };
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	324
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	325 #if defined(__amd64) && !defined(__xpv)
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	326 static iommulib_nexops_t iommulib_nexops = {
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	327 IOMMU_NEXOPS_VERSION,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	328 "Rootnex IOMMU ops Vers 1.1",
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	329 NULL,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	330 rootnex_coredma_allochdl,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	331 rootnex_coredma_freehdl,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	332 rootnex_coredma_bindhdl,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	333 rootnex_coredma_unbindhdl,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	334 rootnex_coredma_reset_cookies,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	335 rootnex_coredma_get_cookies,
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	336 rootnex_coredma_set_cookies,
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	337 rootnex_coredma_clear_cookies,
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	338 rootnex_coredma_get_sleep_flags,
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	339 rootnex_coredma_sync,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	340 rootnex_coredma_win,
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	341 rootnex_coredma_hdl_setprivate,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	342 rootnex_coredma_hdl_getprivate
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	343 };
7617 1e2a36b96537 6748924 fix lint warnings in putback for 6747701 Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7613 diff changeset	344 #endif
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	345
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	346 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	347 * extern hacks
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	348 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	349 extern struct seg_ops segdev_ops;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	350 extern int ignore_hardware_nodes; /* force flag from ddi_impl.c */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	351 #ifdef DDI_MAP_DEBUG
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	352 extern int ddi_map_debug_flag;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	353 #define ddi_map_debug if (ddi_map_debug_flag) prom_printf
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	354 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	355 extern void i86_pp_map(page_t *pp, caddr_t kaddr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	356 extern void i86_va_map(caddr_t vaddr, struct as *asp, caddr_t kaddr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	357 extern int (psm_intr_ops)(dev_info_t , ddi_intr_handle_impl_t *,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	358 psm_intr_op_t, int *);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	359 extern int impl_ddi_sunbus_initchild(dev_info_t *dip);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	360 extern void impl_ddi_sunbus_removechild(dev_info_t *dip);
5251 aad4f9e16063 PSARC/2007/589 dma-max-arch scsi capability mrj parents: 5084 diff changeset	361
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	362 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	363 * Use device arena to use for device control register mappings.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	364 * Various kernel memory walkers (debugger, dtrace) need to know
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	365 * to avoid this address range to prevent undesired device activity.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	366 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	367 extern void *device_arena_alloc(size_t size, int vm_flag);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	368 extern void device_arena_free(void * vaddr, size_t size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	369
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	370
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	371 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	372 * Internal functions
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	373 */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	374 static int rootnex_dma_init();
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	375 static void rootnex_add_props(dev_info_t *);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	376 static int rootnex_ctl_reportdev(dev_info_t *dip);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	377 static struct intrspec rootnex_get_ispec(dev_info_t rdip, int inum);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	378 static int rootnex_map_regspec(ddi_map_req_t mp, caddr_t vaddrp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	379 static int rootnex_unmap_regspec(ddi_map_req_t mp, caddr_t vaddrp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	380 static int rootnex_map_handle(ddi_map_req_t *mp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	381 static void rootnex_clean_dmahdl(ddi_dma_impl_t *hp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	382 static int rootnex_valid_alloc_parms(ddi_dma_attr_t *attr, uint_t maxsegsize);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	383 static int rootnex_valid_bind_parms(ddi_dma_req_t *dmareq,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	384 ddi_dma_attr_t *attr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	385 static void rootnex_get_sgl(ddi_dma_obj_t dmar_object, ddi_dma_cookie_t sgl,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	386 rootnex_sglinfo_t *sglinfo);
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	387 static void rootnex_dvma_get_sgl(ddi_dma_obj_t *dmar_object,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	388 ddi_dma_cookie_t sgl, rootnex_sglinfo_t sglinfo);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	389 static int rootnex_bind_slowpath(ddi_dma_impl_t hp, struct ddi_dma_req dmareq,
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	390 rootnex_dma_t dma, ddi_dma_attr_t attr, ddi_dma_obj_t *dmao, int kmflag);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	391 static int rootnex_setup_copybuf(ddi_dma_impl_t hp, struct ddi_dma_req dmareq,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	392 rootnex_dma_t dma, ddi_dma_attr_t attr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	393 static void rootnex_teardown_copybuf(rootnex_dma_t *dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	394 static int rootnex_setup_windows(ddi_dma_impl_t hp, rootnex_dma_t dma,
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	395 ddi_dma_attr_t attr, ddi_dma_obj_t dmao, int kmflag);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	396 static void rootnex_teardown_windows(rootnex_dma_t *dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	397 static void rootnex_init_win(ddi_dma_impl_t hp, rootnex_dma_t dma,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	398 rootnex_window_t window, ddi_dma_cookie_t cookie, off_t cur_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	399 static void rootnex_setup_cookie(ddi_dma_obj_t *dmar_object,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	400 rootnex_dma_t dma, ddi_dma_cookie_t cookie, off_t cur_offset,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	401 size_t copybuf_used, page_t *cur_pp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	402 static int rootnex_sgllen_window_boundary(ddi_dma_impl_t *hp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	403 rootnex_dma_t dma, rootnex_window_t windowp, ddi_dma_cookie_t cookie,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	404 ddi_dma_attr_t *attr, off_t cur_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	405 static int rootnex_copybuf_window_boundary(ddi_dma_impl_t *hp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	406 rootnex_dma_t dma, rootnex_window_t *windowp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	407 ddi_dma_cookie_t cookie, off_t cur_offset, size_t copybuf_used);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	408 static int rootnex_maxxfer_window_boundary(ddi_dma_impl_t *hp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	409 rootnex_dma_t dma, rootnex_window_t windowp, ddi_dma_cookie_t cookie);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	410 static int rootnex_valid_sync_parms(ddi_dma_impl_t hp, rootnex_window_t win,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	411 off_t offset, size_t size, uint_t cache_flags);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	412 static int rootnex_verify_buffer(rootnex_dma_t *dma);
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	413 static int rootnex_dma_check(dev_info_t dip, const void handle,
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	414 const void comp_addr, const void not_used);
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	415 static boolean_t rootnex_need_bounce_seg(ddi_dma_obj_t *dmar_object,
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	416 rootnex_sglinfo_t *sglinfo);
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	417 static struct as rootnex_get_as(ddi_dma_obj_t dmar_object);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	418
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	419 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	420 * _init()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	421 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	422 */
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	423 int
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	424 _init(void)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	425 {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	426
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	427 rootnex_state = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	428 return (mod_install(&rootnex_modlinkage));
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	429 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	430
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	431
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	432 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	433 * _info()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	434 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	435 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	436 int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	437 _info(struct modinfo *modinfop)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	438 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	439 return (mod_info(&rootnex_modlinkage, modinfop));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	440 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	441
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	442
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	443 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	444 * _fini()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	445 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	446 */
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	447 int
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	448 _fini(void)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	449 {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	450 return (EBUSY);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	451 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	452
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	453
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	454 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	455 * rootnex_attach()
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	456 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	457 */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	458 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	459 rootnex_attach(dev_info_t *dip, ddi_attach_cmd_t cmd)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	460 {
1414 b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	461 int fmcap;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	462 int e;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	463
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	464 switch (cmd) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	465 case DDI_ATTACH:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	466 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	467 case DDI_RESUME:
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	468 #if defined(__amd64) && !defined(__xpv)
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	469 return (immu_unquiesce());
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	470 #else
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	471 return (DDI_SUCCESS);
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	472 #endif
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	473 default:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	474 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	475 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	476
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	477 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	478 * We should only have one instance of rootnex. Save it away since we
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	479 * don't have an easy way to get it back later.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	480 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	481 ASSERT(rootnex_state == NULL);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	482 rootnex_state = kmem_zalloc(sizeof (rootnex_state_t), KM_SLEEP);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	483
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	484 rootnex_state->r_dip = dip;
1414 b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	485 rootnex_state->r_err_ibc = (ddi_iblock_cookie_t)ipltospl(15);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	486 rootnex_state->r_reserved_msg_printed = B_FALSE;
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	487 #ifdef DEBUG
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	488 rootnex_cnt = &rootnex_state->r_counters[0];
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	489 #endif
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	490
1414 b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	491 /*
b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	492 * Set minimum fm capability level for i86pc platforms and then
b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	493 * initialize error handling. Since we're the rootnex, we don't
b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	494 * care what's returned in the fmcap field.
b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	495 */
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	496 ddi_system_fmcap = DDI_FM_EREPORT_CAPABLE \| DDI_FM_ERRCB_CAPABLE \|
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	497 DDI_FM_ACCCHK_CAPABLE \| DDI_FM_DMACHK_CAPABLE;
1414 b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	498 fmcap = ddi_system_fmcap;
b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	499 ddi_fm_init(dip, &fmcap, &rootnex_state->r_err_ibc);
b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	500
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	501 /* initialize DMA related state */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	502 e = rootnex_dma_init();
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	503 if (e != DDI_SUCCESS) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	504 kmem_free(rootnex_state, sizeof (rootnex_state_t));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	505 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	506 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	507
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	508 /* Add static root node properties */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	509 rootnex_add_props(dip);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	510
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	511 /* since we can't call ddi_report_dev() */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	512 cmn_err(CE_CONT, "?root nexus = %s\n", ddi_get_name(dip));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	513
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	514 /* Initialize rootnex event handle */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	515 i_ddi_rootnex_init_events(dip);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	516
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	517 #if defined(__amd64) && !defined(__xpv)
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	518 e = iommulib_nexus_register(dip, &iommulib_nexops,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	519 &rootnex_state->r_iommulib_handle);
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	520
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	521 ASSERT(e == DDI_SUCCESS);
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	522 #endif
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	523
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	524 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	525 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	526
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	527
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	528 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	529 * rootnex_detach()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	530 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	531 */
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	532 /ARGSUSED/
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	533 static int
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	534 rootnex_detach(dev_info_t *dip, ddi_detach_cmd_t cmd)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	535 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	536 switch (cmd) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	537 case DDI_SUSPEND:
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	538 #if defined(__amd64) && !defined(__xpv)
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	539 return (immu_quiesce());
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	540 #else
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	541 return (DDI_SUCCESS);
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	542 #endif
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	543 default:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	544 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	545 }
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	546 /NOTREACHED/
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	547
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	548 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	549
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	550
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	551 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	552 * rootnex_dma_init()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	553 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	554 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	555 /ARGSUSED/
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	556 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	557 rootnex_dma_init()
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	558 {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	559 size_t bufsize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	560
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	561
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	562 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	563 * size of our cookie/window/copybuf state needed in dma bind that we
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	564 * pre-alloc in dma_alloc_handle
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	565 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	566 rootnex_state->r_prealloc_cookies = rootnex_prealloc_cookies;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	567 rootnex_state->r_prealloc_size =
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	568 (rootnex_state->r_prealloc_cookies * sizeof (ddi_dma_cookie_t)) +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	569 (rootnex_prealloc_windows * sizeof (rootnex_window_t)) +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	570 (rootnex_prealloc_copybuf * sizeof (rootnex_pgmap_t));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	571
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	572 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	573 * setup DDI DMA handle kmem cache, align each handle on 64 bytes,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	574 * allocate 16 extra bytes for struct pointer alignment
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	575 * (p->dmai_private & dma->dp_prealloc_buffer)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	576 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	577 bufsize = sizeof (ddi_dma_impl_t) + sizeof (rootnex_dma_t) +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	578 rootnex_state->r_prealloc_size + 0x10;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	579 rootnex_state->r_dmahdl_cache = kmem_cache_create("rootnex_dmahdl",
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	580 bufsize, 64, NULL, NULL, NULL, NULL, NULL, 0);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	581 if (rootnex_state->r_dmahdl_cache == NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	582 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	583 }
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	584
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	585 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	586 * allocate array to track which major numbers we have printed warnings
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	587 * for.
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	588 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	589 rootnex_warn_list = kmem_zalloc(devcnt * sizeof (*rootnex_warn_list),
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	590 KM_SLEEP);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	591
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	592 return (DDI_SUCCESS);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	593 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	594
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	595
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	596 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	597 * rootnex_add_props()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	598 *
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	599 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	600 static void
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	601 rootnex_add_props(dev_info_t *dip)
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	602 {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	603 rootnex_intprop_t *rpp;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	604 int i;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	605
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	606 /* Add static integer/boolean properties to the root node */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	607 rpp = rootnex_intprp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	608 for (i = 0; i < NROOT_INTPROPS; i++) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	609 (void) e_ddi_prop_update_int(DDI_DEV_T_NONE, dip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	610 rpp[i].prop_name, rpp[i].prop_value);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	611 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	612 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	613
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	614
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	615
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	616 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	617 * *************************
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	618 * ctlops related routines
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	619 * *************************
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	620 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	621
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	622 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	623 * rootnex_ctlops()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	624 *
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	625 */
693 1c08294a694e PSARC/2005/561 Obsolete DDI Interrupt Interfaces govinda parents: 683 diff changeset	626 /ARGSUSED/
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	627 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	628 rootnex_ctlops(dev_info_t dip, dev_info_t rdip, ddi_ctl_enum_t ctlop,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	629 void arg, void result)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	630 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	631 int n, *ptr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	632 struct ddi_parent_private_data *pdp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	633
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	634 switch (ctlop) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	635 case DDI_CTLOPS_DMAPMAPC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	636 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	637 * Return 'partial' to indicate that dma mapping
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	638 * has to be done in the main MMU.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	639 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	640 return (DDI_DMA_PARTIAL);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	641
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	642 case DDI_CTLOPS_BTOP:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	643 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	644 * Convert byte count input to physical page units.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	645 * (byte counts that are not a page-size multiple
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	646 * are rounded down)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	647 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	648 (ulong_t )result = btop((ulong_t )arg);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	649 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	650
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	651 case DDI_CTLOPS_PTOB:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	652 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	653 * Convert size in physical pages to bytes
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	654 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	655 (ulong_t )result = ptob((ulong_t )arg);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	656 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	657
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	658 case DDI_CTLOPS_BTOPR:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	659 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	660 * Convert byte count input to physical page units
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	661 * (byte counts that are not a page-size multiple
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	662 * are rounded up)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	663 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	664 (ulong_t )result = btopr((ulong_t )arg);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	665 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	666
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	667 case DDI_CTLOPS_INITCHILD:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	668 return (impl_ddi_sunbus_initchild(arg));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	669
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	670 case DDI_CTLOPS_UNINITCHILD:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	671 impl_ddi_sunbus_removechild(arg);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	672 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	673
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	674 case DDI_CTLOPS_REPORTDEV:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	675 return (rootnex_ctl_reportdev(rdip));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	676
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	677 case DDI_CTLOPS_IOMIN:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	678 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	679 * Nothing to do here but reflect back..
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	680 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	681 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	682
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	683 case DDI_CTLOPS_REGSIZE:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	684 case DDI_CTLOPS_NREGS:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	685 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	686
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	687 case DDI_CTLOPS_SIDDEV:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	688 if (ndi_dev_is_prom_node(rdip))
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	689 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	690 if (ndi_dev_is_persistent_node(rdip))
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	691 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	692 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	693
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	694 case DDI_CTLOPS_POWER:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	695 return ((pm_platform_power)((power_req_t )arg));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	696
693 1c08294a694e PSARC/2005/561 Obsolete DDI Interrupt Interfaces govinda parents: 683 diff changeset	697 case DDI_CTLOPS_RESERVED0: /* Was DDI_CTLOPS_NINTRS, obsolete */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	698 case DDI_CTLOPS_RESERVED1: /* Was DDI_CTLOPS_POKE_INIT, obsolete */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	699 case DDI_CTLOPS_RESERVED2: /* Was DDI_CTLOPS_POKE_FLUSH, obsolete */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	700 case DDI_CTLOPS_RESERVED3: /* Was DDI_CTLOPS_POKE_FINI, obsolete */
693 1c08294a694e PSARC/2005/561 Obsolete DDI Interrupt Interfaces govinda parents: 683 diff changeset	701 case DDI_CTLOPS_RESERVED4: /* Was DDI_CTLOPS_INTR_HILEVEL, obsolete */
1c08294a694e PSARC/2005/561 Obsolete DDI Interrupt Interfaces govinda parents: 683 diff changeset	702 case DDI_CTLOPS_RESERVED5: /* Was DDI_CTLOPS_XLATE_INTRS, obsolete */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	703 if (!rootnex_state->r_reserved_msg_printed) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	704 rootnex_state->r_reserved_msg_printed = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	705 cmn_err(CE_WARN, "Failing ddi_ctlops call(s) for "
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	706 "1 or more reserved/obsolete operations.");
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	707 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	708 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	709
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	710 default:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	711 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	712 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	713 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	714 * The rest are for "hardware" properties
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	715 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	716 if ((pdp = ddi_get_parent_data(rdip)) == NULL)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	717 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	718
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	719 if (ctlop == DDI_CTLOPS_NREGS) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	720 ptr = (int *)result;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	721 *ptr = pdp->par_nreg;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	722 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	723 off_t size = (off_t )result;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	724
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	725 ptr = (int *)arg;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	726 n = *ptr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	727 if (n >= pdp->par_nreg) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	728 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	729 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	730 *size = (off_t)pdp->par_reg[n].regspec_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	731 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	732 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	733 }
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	734
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	735
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	736 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	737 * rootnex_ctl_reportdev()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	738 *
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	739 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	740 static int
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	741 rootnex_ctl_reportdev(dev_info_t *dev)
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	742 {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	743 int i, n, len, f_len = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	744 char *buf;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	745
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	746 buf = kmem_alloc(REPORTDEV_BUFSIZE, KM_SLEEP);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	747 f_len += snprintf(buf, REPORTDEV_BUFSIZE,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	748 "%s%d at root", ddi_driver_name(dev), ddi_get_instance(dev));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	749 len = strlen(buf);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	750
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	751 for (i = 0; i < sparc_pd_getnreg(dev); i++) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	752
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	753 struct regspec *rp = sparc_pd_getreg(dev, i);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	754
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	755 if (i == 0)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	756 f_len += snprintf(buf + len, REPORTDEV_BUFSIZE - len,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	757 ": ");
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	758 else
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	759 f_len += snprintf(buf + len, REPORTDEV_BUFSIZE - len,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	760 " and ");
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	761 len = strlen(buf);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	762
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	763 switch (rp->regspec_bustype) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	764
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	765 case BTEISA:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	766 f_len += snprintf(buf + len, REPORTDEV_BUFSIZE - len,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	767 "%s 0x%x", DEVI_EISA_NEXNAME, rp->regspec_addr);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	768 break;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	769
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	770 case BTISA:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	771 f_len += snprintf(buf + len, REPORTDEV_BUFSIZE - len,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	772 "%s 0x%x", DEVI_ISA_NEXNAME, rp->regspec_addr);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	773 break;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	774
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	775 default:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	776 f_len += snprintf(buf + len, REPORTDEV_BUFSIZE - len,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	777 "space %x offset %x",
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	778 rp->regspec_bustype, rp->regspec_addr);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	779 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	780 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	781 len = strlen(buf);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	782 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	783 for (i = 0, n = sparc_pd_getnintr(dev); i < n; i++) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	784 int pri;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	785
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	786 if (i != 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	787 f_len += snprintf(buf + len, REPORTDEV_BUFSIZE - len,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	788 ",");
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	789 len = strlen(buf);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	790 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	791 pri = INT_IPL(sparc_pd_getintr(dev, i)->intrspec_pri);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	792 f_len += snprintf(buf + len, REPORTDEV_BUFSIZE - len,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	793 " sparc ipl %d", pri);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	794 len = strlen(buf);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	795 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	796 #ifdef DEBUG
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	797 if (f_len + 1 >= REPORTDEV_BUFSIZE) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	798 cmn_err(CE_NOTE, "next message is truncated: "
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	799 "printed length 1024, real length %d", f_len);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	800 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	801 #endif /* DEBUG */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	802 cmn_err(CE_CONT, "?%s\n", buf);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	803 kmem_free(buf, REPORTDEV_BUFSIZE);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	804 return (DDI_SUCCESS);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	805 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	806
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	807
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	808 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	809 * ******************
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	810 * map related code
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	811 * ******************
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	812 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	813
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	814 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	815 * rootnex_map()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	816 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	817 */
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	818 static int
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	819 rootnex_map(dev_info_t dip, dev_info_t rdip, ddi_map_req_t *mp, off_t offset,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	820 off_t len, caddr_t *vaddrp)
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	821 {
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	822 struct regspec *orp = NULL;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	823 struct regspec64 rp = { 0 };
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	824 ddi_map_req_t mr = mp; / Get private copy of request */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	825
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	826 mp = &mr;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	827
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	828 switch (mp->map_op) {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	829 case DDI_MO_MAP_LOCKED:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	830 case DDI_MO_UNMAP:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	831 case DDI_MO_MAP_HANDLE:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	832 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	833 default:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	834 #ifdef DDI_MAP_DEBUG
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	835 cmn_err(CE_WARN, "rootnex_map: unimplemented map op %d.",
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	836 mp->map_op);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	837 #endif /* DDI_MAP_DEBUG */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	838 return (DDI_ME_UNIMPLEMENTED);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	839 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	840
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	841 if (mp->map_flags & DDI_MF_USER_MAPPING) {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	842 #ifdef DDI_MAP_DEBUG
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	843 cmn_err(CE_WARN, "rootnex_map: unimplemented map type: user.");
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	844 #endif /* DDI_MAP_DEBUG */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	845 return (DDI_ME_UNIMPLEMENTED);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	846 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	847
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	848 /*
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	849 * First, we need to get the original regspec out before we convert it
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	850 * to the extended format. If we have a register number, then we need to
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	851 * convert that to a regspec.
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	852 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	853 if (mp->map_type == DDI_MT_RNUMBER) {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	854
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	855 int rnumber = mp->map_obj.rnumber;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	856 #ifdef DDI_MAP_DEBUG
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	857 static char *out_of_range =
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	858 "rootnex_map: Out of range rnumber <%d>, device <%s>";
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	859 #endif /* DDI_MAP_DEBUG */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	860
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	861 orp = i_ddi_rnumber_to_regspec(rdip, rnumber);
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	862 if (orp == NULL) {
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	863 #ifdef DDI_MAP_DEBUG
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	864 cmn_err(CE_WARN, out_of_range, rnumber,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	865 ddi_get_name(rdip));
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	866 #endif /* DDI_MAP_DEBUG */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	867 return (DDI_ME_RNUMBER_RANGE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	868 }
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	869 } else if (!(mp->map_flags & DDI_MF_EXT_REGSPEC)) {
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	870 orp = mp->map_obj.rp;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	871 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	872
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	873 /*
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	874 * Ensure that we are always using a 64-bit extended regspec regardless
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	875 * of what was passed into us. If the child driver is using a 64-bit
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	876 * regspec, then we need to make sure that we copy this to the local
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	877 * regspec64, rp.
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	878 */
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	879 if (orp != NULL) {
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	880 rp.regspec_bustype = orp->regspec_bustype;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	881 rp.regspec_addr = orp->regspec_addr;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	882 rp.regspec_size = orp->regspec_size;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	883 } else {
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	884 struct regspec64 *rp64;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	885 rp64 = (struct regspec64 *)mp->map_obj.rp;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	886 rp = *rp64;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	887 }
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	888
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	889 mp->map_type = DDI_MT_REGSPEC;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	890 mp->map_flags \|= DDI_MF_EXT_REGSPEC;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	891 mp->map_obj.rp = (struct regspec *)&rp;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	892
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	893 /*
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	894 * Adjust offset and length correspnding to called values...
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	895 * XXX: A non-zero length means override the one in the regspec
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	896 * XXX: (regardless of what's in the parent's range?)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	897 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	898
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	899 #ifdef DDI_MAP_DEBUG
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	900 cmn_err(CE_CONT, "rootnex: <%s,%s> <0x%x, 0x%x, 0x%d> offset %d len %d "
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	901 "handle 0x%x\n", ddi_get_name(dip), ddi_get_name(rdip),
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	902 rp.regspec_bustype, rp.regspec_addr, rp.regspec_size, offset,
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	903 len, mp->map_handlep);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	904 #endif /* DDI_MAP_DEBUG */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	905
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	906 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	907 * I/O or memory mapping:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	908 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	909 * <bustype=0, addr=x, len=x>: memory
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	910 * <bustype=1, addr=x, len=x>: i/o
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	911 * <bustype>1, addr=0, len=x>: x86-compatibility i/o
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	912 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	913
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	914 if (rp.regspec_bustype > 1 && rp.regspec_addr != 0) {
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	915 cmn_err(CE_WARN, "<%s,%s> invalid register spec"
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	916 " <0x%" PRIx64 ", 0x%" PRIx64 ", 0x%" PRIx64 ">",
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	917 ddi_get_name(dip), ddi_get_name(rdip), rp.regspec_bustype,
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	918 rp.regspec_addr, rp.regspec_size);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	919 return (DDI_ME_INVAL);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	920 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	921
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	922 if (rp.regspec_bustype > 1 && rp.regspec_addr == 0) {
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	923 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	924 * compatibility i/o mapping
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	925 */
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	926 rp.regspec_bustype += offset;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	927 } else {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	928 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	929 * Normal memory or i/o mapping
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	930 */
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	931 rp.regspec_addr += offset;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	932 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	933
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	934 if (len != 0)
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	935 rp.regspec_size = len;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	936
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	937 #ifdef DDI_MAP_DEBUG
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	938 cmn_err(CE_CONT, " <%s,%s> <0x%" PRIx64 ", 0x%" PRIx64
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	939 ", 0x%" PRId64 "> offset %d len %d handle 0x%x\n",
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	940 ddi_get_name(dip), ddi_get_name(rdip), rp.regspec_bustype,
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	941 rp.regspec_addr, rp.regspec_size, offset, len, mp->map_handlep);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	942 #endif /* DDI_MAP_DEBUG */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	943
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	944
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	945 /*
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	946 * The x86 root nexus does not have any notion of valid ranges of
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	947 * addresses. Its children have valid ranges, but because there are none
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	948 * for the nexus, we don't need to call i_ddi_apply_range(). Verify
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	949 * that is the case.
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	950 */
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	951 ASSERT0(sparc_pd_getnrng(dip));
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	952
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	953 switch (mp->map_op) {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	954 case DDI_MO_MAP_LOCKED:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	955
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	956 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	957 * Set up the locked down kernel mapping to the regspec...
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	958 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	959
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	960 return (rootnex_map_regspec(mp, vaddrp));
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	961
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	962 case DDI_MO_UNMAP:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	963
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	964 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	965 * Release mapping...
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	966 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	967
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	968 return (rootnex_unmap_regspec(mp, vaddrp));
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	969
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	970 case DDI_MO_MAP_HANDLE:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	971
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	972 return (rootnex_map_handle(mp));
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	973
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	974 default:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	975 return (DDI_ME_UNIMPLEMENTED);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	976 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	977 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	978
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	979
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	980 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	981 * rootnex_map_fault()
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	982 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	983 * fault in mappings for requestors
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	984 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	985 /ARGSUSED/
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	986 static int
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	987 rootnex_map_fault(dev_info_t dip, dev_info_t rdip, struct hat *hat,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	988 struct seg seg, caddr_t addr, struct devpage dp, pfn_t pfn, uint_t prot,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	989 uint_t lock)
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	990 {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	991
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	992 #ifdef DDI_MAP_DEBUG
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	993 ddi_map_debug("rootnex_map_fault: address <%x> pfn <%x>", addr, pfn);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	994 ddi_map_debug(" Seg <%s>\n",
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	995 seg->s_ops == &segdev_ops ? "segdev" :
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	996 seg == &kvseg ? "segkmem" : "NONE!");
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	997 #endif /* DDI_MAP_DEBUG */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	998
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	999 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1000 * This is all terribly broken, but it is a start
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1001 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1002 * XXX Note that this test means that segdev_ops
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1003 * must be exported from seg_dev.c.
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1004 * XXX What about devices with their own segment drivers?
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1005 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1006 if (seg->s_ops == &segdev_ops) {
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1007 struct segdev_data sdp = (struct segdev_data )seg->s_data;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1008
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1009 if (hat == NULL) {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1010 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1011 * This is one plausible interpretation of
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1012 * a null hat i.e. use the first hat on the
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1013 * address space hat list which by convention is
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1014 * the hat of the system MMU. At alternative
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1015 * would be to panic .. this might well be better ..
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1016 */
15433 fb195d6afcd7 6514 AS_* lock macros simplification Josef 'Jeff' Sipek <jeffpc@josefsipek.net> parents: 14707 diff changeset	1017 ASSERT(AS_READ_HELD(seg->s_as));
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1018 hat = seg->s_as->a_hat;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1019 cmn_err(CE_NOTE, "rootnex_map_fault: nil hat");
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1020 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1021 hat_devload(hat, addr, MMU_PAGESIZE, pfn, prot \| sdp->hat_attr,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1022 (lock ? HAT_LOAD_LOCK : HAT_LOAD));
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1023 } else if (seg == &kvseg && dp == NULL) {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1024 hat_devload(kas.a_hat, addr, MMU_PAGESIZE, pfn, prot,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1025 HAT_LOAD_LOCK);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1026 } else
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1027 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1028 return (DDI_SUCCESS);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1029 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1030
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1031
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1032 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1033 rootnex_map_regspec(ddi_map_req_t mp, caddr_t vaddrp)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1034 {
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1035 rootnex_addr_t rbase;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1036 void *cvaddr;
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1037 uint64_t npages, pgoffset;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1038 struct regspec64 *rp;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1039 ddi_acc_hdl_t *hp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1040 ddi_acc_impl_t *ap;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1041 uint_t hat_acc_flags;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1042 paddr_t pbase;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1043
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1044 ASSERT(mp->map_flags & DDI_MF_EXT_REGSPEC);
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1045 rp = (struct regspec64 *)mp->map_obj.rp;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1046 hp = mp->map_handlep;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1047
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1048 #ifdef DDI_MAP_DEBUG
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1049 ddi_map_debug(
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1050 "rootnex_map_regspec: <0x%x 0x%x 0x%x> handle 0x%x\n",
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1051 rp->regspec_bustype, rp->regspec_addr,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1052 rp->regspec_size, mp->map_handlep);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1053 #endif /* DDI_MAP_DEBUG */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1054
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1055 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1056 * I/O or memory mapping
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1057 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1058 * <bustype=0, addr=x, len=x>: memory
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1059 * <bustype=1, addr=x, len=x>: i/o
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1060 * <bustype>1, addr=0, len=x>: x86-compatibility i/o
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1061 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1062
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1063 if (rp->regspec_bustype > 1 && rp->regspec_addr != 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1064 cmn_err(CE_WARN, "rootnex: invalid register spec"
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1065 " <0x%" PRIx64 ", 0x%" PRIx64", 0x%" PRIx64">",
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1066 rp->regspec_bustype, rp->regspec_addr, rp->regspec_size);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1067 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1068 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1069
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1070 if (rp->regspec_bustype != 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1071 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1072 * I/O space - needs a handle.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1073 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1074 if (hp == NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1075 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1076 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1077 ap = (ddi_acc_impl_t *)hp->ah_platform_private;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1078 ap->ahi_acc_attr \|= DDI_ACCATTR_IO_SPACE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1079 impl_acc_hdl_init(hp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1080
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1081 if (mp->map_flags & DDI_MF_DEVICE_MAPPING) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1082 #ifdef DDI_MAP_DEBUG
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1083 ddi_map_debug("rootnex_map_regspec: mmap() "
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1084 "to I/O space is not supported.\n");
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1085 #endif /* DDI_MAP_DEBUG */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1086 return (DDI_ME_INVAL);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1087 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1088 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1089 * 1275-compliant vs. compatibility i/o mapping
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1090 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1091 *vaddrp =
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1092 (rp->regspec_bustype > 1 && rp->regspec_addr == 0) ?
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1093 ((caddr_t)(uintptr_t)rp->regspec_bustype) :
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1094 ((caddr_t)(uintptr_t)rp->regspec_addr);
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1095 #ifdef __xpv
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1096 if (DOMAIN_IS_INITDOMAIN(xen_info)) {
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1097 hp->ah_pfn = xen_assign_pfn(
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1098 mmu_btop((ulong_t)rp->regspec_addr &
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1099 MMU_PAGEMASK));
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1100 } else {
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1101 hp->ah_pfn = mmu_btop(
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1102 (ulong_t)rp->regspec_addr & MMU_PAGEMASK);
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1103 }
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1104 #else
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	1105 hp->ah_pfn = mmu_btop((ulong_t)rp->regspec_addr &
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1106 MMU_PAGEMASK);
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1107 #endif
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	1108 hp->ah_pnum = mmu_btopr(rp->regspec_size +
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	1109 (ulong_t)rp->regspec_addr & MMU_PAGEOFFSET);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1110 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1111
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1112 #ifdef DDI_MAP_DEBUG
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1113 ddi_map_debug(
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1114 "rootnex_map_regspec: \"Mapping\" %d bytes I/O space at 0x%x\n",
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1115 rp->regspec_size, *vaddrp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1116 #endif /* DDI_MAP_DEBUG */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1117 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1118 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1119
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1120 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1121 * Memory space
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1122 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1123
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1124 if (hp != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1125 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1126 * hat layer ignores
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1127 * hp->ah_acc.devacc_attr_endian_flags.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1128 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1129 switch (hp->ah_acc.devacc_attr_dataorder) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1130 case DDI_STRICTORDER_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1131 hat_acc_flags = HAT_STRICTORDER;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1132 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1133 case DDI_UNORDERED_OK_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1134 hat_acc_flags = HAT_UNORDERED_OK;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1135 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1136 case DDI_MERGING_OK_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1137 hat_acc_flags = HAT_MERGING_OK;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1138 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1139 case DDI_LOADCACHING_OK_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1140 hat_acc_flags = HAT_LOADCACHING_OK;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1141 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1142 case DDI_STORECACHING_OK_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1143 hat_acc_flags = HAT_STORECACHING_OK;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1144 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1145 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1146 ap = (ddi_acc_impl_t *)hp->ah_platform_private;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1147 ap->ahi_acc_attr \|= DDI_ACCATTR_CPU_VADDR;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1148 impl_acc_hdl_init(hp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1149 hp->ah_hat_flags = hat_acc_flags;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1150 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1151 hat_acc_flags = HAT_STRICTORDER;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1152 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1153
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1154 rbase = (rootnex_addr_t)(rp->regspec_addr & MMU_PAGEMASK);
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1155 #ifdef __xpv
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1156 /*
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1157 * If we're dom0, we're using a real device so we need to translate
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1158 * the MA to a PA.
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1159 */
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1160 if (DOMAIN_IS_INITDOMAIN(xen_info)) {
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1161 pbase = pfn_to_pa(xen_assign_pfn(mmu_btop(rbase)));
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1162 } else {
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1163 pbase = rbase;
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1164 }
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1165 #else
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1166 pbase = rbase;
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1167 #endif
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1168 pgoffset = (ulong_t)rp->regspec_addr & MMU_PAGEOFFSET;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1169
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1170 if (rp->regspec_size == 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1171 #ifdef DDI_MAP_DEBUG
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1172 ddi_map_debug("rootnex_map_regspec: zero regspec_size\n");
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1173 #endif /* DDI_MAP_DEBUG */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1174 return (DDI_ME_INVAL);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1175 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1176
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1177 if (mp->map_flags & DDI_MF_DEVICE_MAPPING) {
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1178 /* extra cast to make gcc happy */
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1179 *vaddrp = (caddr_t)((uintptr_t)mmu_btop(pbase));
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1180 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1181 npages = mmu_btopr(rp->regspec_size + pgoffset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1182
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1183 #ifdef DDI_MAP_DEBUG
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1184 ddi_map_debug("rootnex_map_regspec: Mapping %d pages "
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1185 "physical %llx", npages, pbase);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1186 #endif /* DDI_MAP_DEBUG */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1187
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1188 cvaddr = device_arena_alloc(ptob(npages), VM_NOSLEEP);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1189 if (cvaddr == NULL)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1190 return (DDI_ME_NORESOURCES);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1191
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1192 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1193 * Now map in the pages we've allocated...
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1194 */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1195 hat_devload(kas.a_hat, cvaddr, mmu_ptob(npages),
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1196 mmu_btop(pbase), mp->map_prot \| hat_acc_flags,
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1197 HAT_LOAD_LOCK);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1198 *vaddrp = (caddr_t)cvaddr + pgoffset;
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	1199
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	1200 /* save away pfn and npages for FMA */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	1201 hp = mp->map_handlep;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	1202 if (hp) {
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1203 hp->ah_pfn = mmu_btop(pbase);
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	1204 hp->ah_pnum = npages;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	1205 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1206 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1207
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1208 #ifdef DDI_MAP_DEBUG
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1209 ddi_map_debug("at virtual 0x%x\n", *vaddrp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1210 #endif /* DDI_MAP_DEBUG */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1211 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1212 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1213
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1214
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1215 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1216 rootnex_unmap_regspec(ddi_map_req_t mp, caddr_t vaddrp)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1217 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1218 caddr_t addr = (caddr_t)*vaddrp;
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1219 uint64_t npages, pgoffset;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1220 struct regspec64 *rp;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1221
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1222 if (mp->map_flags & DDI_MF_DEVICE_MAPPING)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1223 return (0);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1224
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1225 ASSERT(mp->map_flags & DDI_MF_EXT_REGSPEC);
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1226 rp = (struct regspec64 *)mp->map_obj.rp;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1227
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1228 if (rp->regspec_size == 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1229 #ifdef DDI_MAP_DEBUG
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1230 ddi_map_debug("rootnex_unmap_regspec: zero regspec_size\n");
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1231 #endif /* DDI_MAP_DEBUG */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1232 return (DDI_ME_INVAL);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1233 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1234
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1235 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1236 * I/O or memory mapping:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1237 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1238 * <bustype=0, addr=x, len=x>: memory
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1239 * <bustype=1, addr=x, len=x>: i/o
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1240 * <bustype>1, addr=0, len=x>: x86-compatibility i/o
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1241 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1242 if (rp->regspec_bustype != 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1243 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1244 * This is I/O space, which requires no particular
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1245 * processing on unmap since it isn't mapped in the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1246 * first place.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1247 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1248 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1249 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1250
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1251 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1252 * Memory space
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1253 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1254 pgoffset = (uintptr_t)addr & MMU_PAGEOFFSET;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1255 npages = mmu_btopr(rp->regspec_size + pgoffset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1256 hat_unload(kas.a_hat, addr - pgoffset, ptob(npages), HAT_UNLOAD_UNLOCK);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1257 device_arena_free(addr - pgoffset, ptob(npages));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1258
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1259 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1260 * Destroy the pointer - the mapping has logically gone
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1261 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1262 *vaddrp = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1263
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1264 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1265 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1266
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1267 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1268 rootnex_map_handle(ddi_map_req_t *mp)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1269 {
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1270 rootnex_addr_t rbase;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1271 ddi_acc_hdl_t *hp;
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1272 uint64_t pgoffset;
42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1273 struct regspec64 *rp;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1274 paddr_t pbase;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1275
16883 42631d95c231 8702 PCI addresses with physaddr > 0xffffffff can't be mapped in Robert Mustacchi <rm@joyent.com> parents: 16727 diff changeset	1276 rp = (struct regspec64 *)mp->map_obj.rp;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1277
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1278 #ifdef DDI_MAP_DEBUG
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1279 ddi_map_debug(
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1280 "rootnex_map_handle: <0x%x 0x%x 0x%x> handle 0x%x\n",
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1281 rp->regspec_bustype, rp->regspec_addr,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1282 rp->regspec_size, mp->map_handlep);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1283 #endif /* DDI_MAP_DEBUG */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1284
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1285 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1286 * I/O or memory mapping:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1287 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1288 * <bustype=0, addr=x, len=x>: memory
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1289 * <bustype=1, addr=x, len=x>: i/o
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1290 * <bustype>1, addr=0, len=x>: x86-compatibility i/o
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1291 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1292 if (rp->regspec_bustype != 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1293 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1294 * This refers to I/O space, and we don't support "mapping"
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1295 * I/O space to a user.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1296 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1297 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1298 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1299
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1300 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1301 * Set up the hat_flags for the mapping.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1302 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1303 hp = mp->map_handlep;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1304
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1305 switch (hp->ah_acc.devacc_attr_endian_flags) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1306 case DDI_NEVERSWAP_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1307 hp->ah_hat_flags = HAT_NEVERSWAP \| HAT_STRICTORDER;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1308 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1309 case DDI_STRUCTURE_LE_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1310 hp->ah_hat_flags = HAT_STRUCTURE_LE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1311 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1312 case DDI_STRUCTURE_BE_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1313 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1314 default:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1315 return (DDI_REGS_ACC_CONFLICT);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1316 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1317
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1318 switch (hp->ah_acc.devacc_attr_dataorder) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1319 case DDI_STRICTORDER_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1320 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1321 case DDI_UNORDERED_OK_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1322 hp->ah_hat_flags \|= HAT_UNORDERED_OK;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1323 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1324 case DDI_MERGING_OK_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1325 hp->ah_hat_flags \|= HAT_MERGING_OK;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1326 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1327 case DDI_LOADCACHING_OK_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1328 hp->ah_hat_flags \|= HAT_LOADCACHING_OK;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1329 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1330 case DDI_STORECACHING_OK_ACC:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1331 hp->ah_hat_flags \|= HAT_STORECACHING_OK;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1332 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1333 default:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1334 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1335 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1336
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1337 rbase = (rootnex_addr_t)rp->regspec_addr &
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1338 (~(rootnex_addr_t)MMU_PAGEOFFSET);
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1339 pgoffset = (ulong_t)rp->regspec_addr & MMU_PAGEOFFSET;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1340
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1341 if (rp->regspec_size == 0)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1342 return (DDI_ME_INVAL);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1343
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1344 #ifdef __xpv
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1345 /*
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1346 * If we're dom0, we're using a real device so we need to translate
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1347 * the MA to a PA.
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1348 */
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1349 if (DOMAIN_IS_INITDOMAIN(xen_info)) {
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1350 pbase = pfn_to_pa(xen_assign_pfn(mmu_btop(rbase))) \|
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1351 (rbase & MMU_PAGEOFFSET);
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1352 } else {
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1353 pbase = rbase;
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1354 }
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1355 #else
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1356 pbase = rbase;
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1357 #endif
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1358
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	1359 hp->ah_pfn = mmu_btop(pbase);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1360 hp->ah_pnum = mmu_btopr(rp->regspec_size + pgoffset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1361
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1362 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1363 }
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1364
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1365
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1366
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1367 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1368 * ************************
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1369 * interrupt related code
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1370 * ************************
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1371 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1372
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1373 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1374 * rootnex_intr_ops()
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1375 * bus_intr_op() function for interrupt support
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1376 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1377 /* ARGSUSED */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1378 static int
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1379 rootnex_intr_ops(dev_info_t pdip, dev_info_t rdip, ddi_intr_op_t intr_op,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1380 ddi_intr_handle_impl_t hdlp, void result)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1381 {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1382 struct intrspec *ispec;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1383
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1384 DDI_INTR_NEXDBG((CE_CONT,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1385 "rootnex_intr_ops: pdip = %p, rdip = %p, intr_op = %x, hdlp = %p\n",
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1386 (void )pdip, (void )rdip, intr_op, (void *)hdlp));
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1387
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1388 /* Process the interrupt operation */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1389 switch (intr_op) {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1390 case DDI_INTROP_GETCAP:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1391 /* First check with pcplusmp */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1392 if (psm_intr_ops == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1393 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1394
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1395 if ((*psm_intr_ops)(rdip, hdlp, PSM_INTR_OP_GET_CAP, result)) {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1396 (int )result = 0;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1397 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1398 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1399 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1400 case DDI_INTROP_SETCAP:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1401 if (psm_intr_ops == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1402 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1403
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1404 if ((*psm_intr_ops)(rdip, hdlp, PSM_INTR_OP_SET_CAP, result))
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1405 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1406 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1407 case DDI_INTROP_ALLOC:
12683 92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1408 ASSERT(hdlp->ih_type == DDI_INTR_TYPE_FIXED);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1409 return (rootnex_alloc_intr_fixed(rdip, hdlp, result));
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1410 case DDI_INTROP_FREE:
12683 92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1411 ASSERT(hdlp->ih_type == DDI_INTR_TYPE_FIXED);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1412 return (rootnex_free_intr_fixed(rdip, hdlp));
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1413 case DDI_INTROP_GETPRI:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1414 if ((ispec = rootnex_get_ispec(rdip, hdlp->ih_inum)) == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1415 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1416 (int )result = ispec->intrspec_pri;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1417 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1418 case DDI_INTROP_SETPRI:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1419 /* Validate the interrupt priority passed to us */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1420 if ((int )result > LOCK_LEVEL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1421 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1422
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1423 /* Ensure that PSM is all initialized and ispec is ok */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1424 if ((psm_intr_ops == NULL) \|\|
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1425 ((ispec = rootnex_get_ispec(rdip, hdlp->ih_inum)) == NULL))
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1426 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1427
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1428 /* Change the priority */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1429 if ((*psm_intr_ops)(rdip, hdlp, PSM_INTR_OP_SET_PRI, result) ==
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1430 PSM_FAILURE)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1431 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1432
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1433 /* update the ispec with the new priority */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1434 ispec->intrspec_pri = (int )result;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1435 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1436 case DDI_INTROP_ADDISR:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1437 if ((ispec = rootnex_get_ispec(rdip, hdlp->ih_inum)) == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1438 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1439 ispec->intrspec_func = hdlp->ih_cb_func;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1440 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1441 case DDI_INTROP_REMISR:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1442 if ((ispec = rootnex_get_ispec(rdip, hdlp->ih_inum)) == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1443 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1444 ispec->intrspec_func = (uint_t (*)()) 0;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1445 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1446 case DDI_INTROP_ENABLE:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1447 if ((ispec = rootnex_get_ispec(rdip, hdlp->ih_inum)) == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1448 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1449
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1450 /* Call psmi to translate irq with the dip */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1451 if (psm_intr_ops == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1452 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1453
916 ed14fe603852 PSARC/2005/584 PSM_INTR_OPS extensions schwartz parents: 693 diff changeset	1454 ((ihdl_plat_t *)hdlp->ih_private)->ip_ispecp = ispec;
11465 ec77021cc782 6853902 MP system panics due to race between add_avintr and rem_avintr Guoli Shu<Kerry.Shu@Sun.COM> parents: 10902 diff changeset	1455 if ((*psm_intr_ops)(rdip, hdlp, PSM_INTR_OP_XLATE_VECTOR,
ec77021cc782 6853902 MP system panics due to race between add_avintr and rem_avintr Guoli Shu<Kerry.Shu@Sun.COM> parents: 10902 diff changeset	1456 (int *)&hdlp->ih_vector) == PSM_FAILURE)
ec77021cc782 6853902 MP system panics due to race between add_avintr and rem_avintr Guoli Shu<Kerry.Shu@Sun.COM> parents: 10902 diff changeset	1457 return (DDI_FAILURE);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1458
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1459 /* Add the interrupt handler */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1460 if (!add_avintr((void *)hdlp, ispec->intrspec_pri,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1461 hdlp->ih_cb_func, DEVI(rdip)->devi_name, hdlp->ih_vector,
916 ed14fe603852 PSARC/2005/584 PSM_INTR_OPS extensions schwartz parents: 693 diff changeset	1462 hdlp->ih_cb_arg1, hdlp->ih_cb_arg2, NULL, rdip))
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1463 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1464 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1465 case DDI_INTROP_DISABLE:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1466 if ((ispec = rootnex_get_ispec(rdip, hdlp->ih_inum)) == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1467 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1468
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1469 /* Call psm_ops() to translate irq with the dip */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1470 if (psm_intr_ops == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1471 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1472
916 ed14fe603852 PSARC/2005/584 PSM_INTR_OPS extensions schwartz parents: 693 diff changeset	1473 ((ihdl_plat_t *)hdlp->ih_private)->ip_ispecp = ispec;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1474 (void) (*psm_intr_ops)(rdip, hdlp,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1475 PSM_INTR_OP_XLATE_VECTOR, (int *)&hdlp->ih_vector);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1476
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1477 /* Remove the interrupt handler */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1478 rem_avintr((void *)hdlp, ispec->intrspec_pri,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1479 hdlp->ih_cb_func, hdlp->ih_vector);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1480 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1481 case DDI_INTROP_SETMASK:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1482 if (psm_intr_ops == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1483 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1484
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1485 if ((*psm_intr_ops)(rdip, hdlp, PSM_INTR_OP_SET_MASK, NULL))
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1486 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1487 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1488 case DDI_INTROP_CLRMASK:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1489 if (psm_intr_ops == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1490 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1491
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1492 if ((*psm_intr_ops)(rdip, hdlp, PSM_INTR_OP_CLEAR_MASK, NULL))
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1493 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1494 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1495 case DDI_INTROP_GETPENDING:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1496 if (psm_intr_ops == NULL)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1497 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1498
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1499 if ((*psm_intr_ops)(rdip, hdlp, PSM_INTR_OP_GET_PENDING,
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1500 result)) {
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1501 (int )result = 0;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1502 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1503 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1504 break;
2580 f32d67e0b369 6460876 ddi_intr_get_navail(9f): fix functionality across platforms and other bugs anish parents: 1900 diff changeset	1505 case DDI_INTROP_NAVAIL:
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1506 case DDI_INTROP_NINTRS:
2580 f32d67e0b369 6460876 ddi_intr_get_navail(9f): fix functionality across platforms and other bugs anish parents: 1900 diff changeset	1507 (int )result = i_ddi_get_intx_nintrs(rdip);
f32d67e0b369 6460876 ddi_intr_get_navail(9f): fix functionality across platforms and other bugs anish parents: 1900 diff changeset	1508 if ((int )result == 0) {
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1509 /*
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1510 * Special case for 'pcic' driver' only. This driver
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1511 * driver is a child of 'isa' and 'rootnex' drivers.
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1512 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1513 * See detailed comments on this in the function
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1514 * rootnex_get_ispec().
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1515 *
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1516 * Children of 'pcic' send 'NINITR' request all the
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1517 * way to rootnex driver. But, the 'pdp->par_nintr'
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1518 * field may not initialized. So, we fake it here
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1519 * to return 1 (a la what PCMCIA nexus does).
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1520 */
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1521 if (strcmp(ddi_get_name(rdip), "pcic") == 0)
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1522 (int )result = 1;
2580 f32d67e0b369 6460876 ddi_intr_get_navail(9f): fix functionality across platforms and other bugs anish parents: 1900 diff changeset	1523 else
f32d67e0b369 6460876 ddi_intr_get_navail(9f): fix functionality across platforms and other bugs anish parents: 1900 diff changeset	1524 return (DDI_FAILURE);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1525 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1526 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1527 case DDI_INTROP_SUPPORTED_TYPES:
2580 f32d67e0b369 6460876 ddi_intr_get_navail(9f): fix functionality across platforms and other bugs anish parents: 1900 diff changeset	1528 (int )result = DDI_INTR_TYPE_FIXED; /* Always ... */
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1529 break;
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1530 default:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1531 return (DDI_FAILURE);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1532 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1533
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1534 return (DDI_SUCCESS);
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1535 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1536
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1537
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1538 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1539 * rootnex_get_ispec()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1540 * convert an interrupt number to an interrupt specification.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1541 * The interrupt number determines which interrupt spec will be
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1542 * returned if more than one exists.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1543 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1544 * Look into the parent private data area of the 'rdip' to find out
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1545 * the interrupt specification. First check to make sure there is
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1546 * one that matchs "inumber" and then return a pointer to it.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1547 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1548 * Return NULL if one could not be found.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1549 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1550 * NOTE: This is needed for rootnex_intr_ops()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1551 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1552 static struct intrspec *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1553 rootnex_get_ispec(dev_info_t *rdip, int inum)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1554 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1555 struct ddi_parent_private_data *pdp = ddi_get_parent_data(rdip);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1556
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1557 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1558 * Special case handling for drivers that provide their own
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1559 * intrspec structures instead of relying on the DDI framework.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1560 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1561 * A broken hardware driver in ON could potentially provide its
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1562 * own intrspec structure, instead of relying on the hardware.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1563 * If these drivers are children of 'rootnex' then we need to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1564 * continue to provide backward compatibility to them here.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1565 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1566 * Following check is a special case for 'pcic' driver which
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1567 * was found to have broken hardwre andby provides its own intrspec.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1568 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1569 * Verbatim comments from this driver are shown here:
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1570 * "Don't use the ddi_add_intr since we don't have a
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1571 * default intrspec in all cases."
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1572 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1573 * Since an 'ispec' may not be always created for it,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1574 * check for that and create one if so.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1575 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1576 * NOTE: Currently 'pcic' is the only driver found to do this.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1577 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1578 if (!pdp->par_intr && strcmp(ddi_get_name(rdip), "pcic") == 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1579 pdp->par_nintr = 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1580 pdp->par_intr = kmem_zalloc(sizeof (struct intrspec) *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1581 pdp->par_nintr, KM_SLEEP);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1582 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1583
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1584 /* Validate the interrupt number */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1585 if (inum >= pdp->par_nintr)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1586 return (NULL);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1587
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1588 /* Get the interrupt structure pointer and return that */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1589 return ((struct intrspec *)&pdp->par_intr[inum]);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1590 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1591
12683 92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1592 /*
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1593 * Allocate interrupt vector for FIXED (legacy) type.
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1594 */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1595 static int
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1596 rootnex_alloc_intr_fixed(dev_info_t rdip, ddi_intr_handle_impl_t hdlp,
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1597 void *result)
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1598 {
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1599 struct intrspec *ispec;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1600 ddi_intr_handle_impl_t info_hdl;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1601 int ret;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1602 int free_phdl = 0;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1603 apic_get_type_t type_info;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1604
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1605 if (psm_intr_ops == NULL)
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1606 return (DDI_FAILURE);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1607
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1608 if ((ispec = rootnex_get_ispec(rdip, hdlp->ih_inum)) == NULL)
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1609 return (DDI_FAILURE);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1610
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1611 /*
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1612 * If the PSM module is "APIX" then pass the request for it
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1613 * to allocate the vector now.
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1614 */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1615 bzero(&info_hdl, sizeof (ddi_intr_handle_impl_t));
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1616 info_hdl.ih_private = &type_info;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1617 if ((*psm_intr_ops)(NULL, &info_hdl, PSM_INTR_OP_APIC_TYPE, NULL) ==
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1618 PSM_SUCCESS && strcmp(type_info.avgi_type, APIC_APIX_NAME) == 0) {
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1619 if (hdlp->ih_private == NULL) { /* allocate phdl structure */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1620 free_phdl = 1;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1621 i_ddi_alloc_intr_phdl(hdlp);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1622 }
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1623 ((ihdl_plat_t *)hdlp->ih_private)->ip_ispecp = ispec;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1624 ret = (*psm_intr_ops)(rdip, hdlp,
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1625 PSM_INTR_OP_ALLOC_VECTORS, result);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1626 if (free_phdl) { /* free up the phdl structure */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1627 free_phdl = 0;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1628 i_ddi_free_intr_phdl(hdlp);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1629 hdlp->ih_private = NULL;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1630 }
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1631 } else {
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1632 /*
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1633 * No APIX module; fall back to the old scheme where the
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1634 * interrupt vector is allocated during ddi_enable_intr() call.
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1635 */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1636 hdlp->ih_pri = ispec->intrspec_pri;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1637 (int )result = hdlp->ih_scratch1;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1638 ret = DDI_SUCCESS;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1639 }
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1640
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1641 return (ret);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1642 }
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1643
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1644 /*
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1645 * Free up interrupt vector for FIXED (legacy) type.
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1646 */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1647 static int
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1648 rootnex_free_intr_fixed(dev_info_t rdip, ddi_intr_handle_impl_t hdlp)
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1649 {
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1650 struct intrspec *ispec;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1651 struct ddi_parent_private_data *pdp;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1652 ddi_intr_handle_impl_t info_hdl;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1653 int ret;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1654 apic_get_type_t type_info;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1655
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1656 if (psm_intr_ops == NULL)
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1657 return (DDI_FAILURE);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1658
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1659 /*
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1660 * If the PSM module is "APIX" then pass the request for it
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1661 * to free up the vector now.
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1662 */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1663 bzero(&info_hdl, sizeof (ddi_intr_handle_impl_t));
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1664 info_hdl.ih_private = &type_info;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1665 if ((*psm_intr_ops)(NULL, &info_hdl, PSM_INTR_OP_APIC_TYPE, NULL) ==
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1666 PSM_SUCCESS && strcmp(type_info.avgi_type, APIC_APIX_NAME) == 0) {
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1667 if ((ispec = rootnex_get_ispec(rdip, hdlp->ih_inum)) == NULL)
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1668 return (DDI_FAILURE);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1669 ((ihdl_plat_t *)hdlp->ih_private)->ip_ispecp = ispec;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1670 ret = (*psm_intr_ops)(rdip, hdlp,
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1671 PSM_INTR_OP_FREE_VECTORS, NULL);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1672 } else {
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1673 /*
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1674 * No APIX module; fall back to the old scheme where
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1675 * the interrupt vector was already freed during
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1676 * ddi_disable_intr() call.
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1677 */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1678 ret = DDI_SUCCESS;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1679 }
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1680
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1681 pdp = ddi_get_parent_data(rdip);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1682
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1683 /*
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1684 * Special case for 'pcic' driver' only.
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1685 * If an intrspec was created for it, clean it up here
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1686 * See detailed comments on this in the function
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1687 * rootnex_get_ispec().
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1688 */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1689 if (pdp->par_intr && strcmp(ddi_get_name(rdip), "pcic") == 0) {
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1690 kmem_free(pdp->par_intr, sizeof (struct intrspec) *
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1691 pdp->par_nintr);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1692 /*
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1693 * Set it to zero; so that
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1694 * DDI framework doesn't free it again
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1695 */
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1696 pdp->par_intr = NULL;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1697 pdp->par_nintr = 0;
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1698 }
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1699
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1700 return (ret);
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1701 }
92e6427b7b70 PSARC/2009/505 IRM Framework Extension(s) Jimmy Vetayases <Jimmy.Vetayases@oracle.com> parents: 12118 diff changeset	1702
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1703
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1704 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1705 * ******************
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1706 * dma related code
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1707 * ******************
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1708 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1709
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1710 /ARGSUSED/
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1711 static int
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1712 rootnex_coredma_allochdl(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1713 ddi_dma_attr_t attr, int (waitfp)(caddr_t), caddr_t arg,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1714 ddi_dma_handle_t *handlep)
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1715 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1716 uint64_t maxsegmentsize_ll;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1717 uint_t maxsegmentsize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1718 ddi_dma_impl_t *hp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1719 rootnex_dma_t *dma;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1720 uint64_t count_max;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1721 uint64_t seg;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1722 int kmflag;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1723 int e;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1724
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1725
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1726 /* convert our sleep flags */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1727 if (waitfp == DDI_DMA_SLEEP) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1728 kmflag = KM_SLEEP;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1729 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1730 kmflag = KM_NOSLEEP;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1731 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1732
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1733 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1734 * We try to do only one memory allocation here. We'll do a little
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1735 * pointer manipulation later. If the bind ends up taking more than
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1736 * our prealloc's space, we'll have to allocate more memory in the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1737 * bind operation. Not great, but much better than before and the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1738 * best we can do with the current bind interfaces.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1739 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1740 hp = kmem_cache_alloc(rootnex_state->r_dmahdl_cache, kmflag);
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1741 if (hp == NULL)
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1742 return (DDI_DMA_NORESOURCES);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1743
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1744 /* Do our pointer manipulation now, align the structures */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1745 hp->dmai_private = (void *)(((uintptr_t)hp +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1746 (uintptr_t)sizeof (ddi_dma_impl_t) + 0x7) & ~0x7);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1747 dma = (rootnex_dma_t *)hp->dmai_private;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1748 dma->dp_prealloc_buffer = (uchar_t *)(((uintptr_t)dma +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1749 sizeof (rootnex_dma_t) + 0x7) & ~0x7);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1750
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1751 /* setup the handle */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1752 rootnex_clean_dmahdl(hp);
12027 6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	1753 hp->dmai_error.err_fep = NULL;
6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	1754 hp->dmai_error.err_cf = NULL;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1755 dma->dp_dip = rdip;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1756 dma->dp_sglinfo.si_flags = attr->dma_attr_flags;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1757 dma->dp_sglinfo.si_min_addr = attr->dma_attr_addr_lo;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1758
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1759 /*
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1760 * The BOUNCE_ON_SEG workaround is not needed when an IOMMU
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1761 * is being used. Set the upper limit to the seg value.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1762 * There will be enough DVMA space to always get addresses
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1763 * that will match the constraints.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1764 */
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1765 if (IOMMU_USED(rdip) &&
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1766 (attr->dma_attr_flags & _DDI_DMA_BOUNCE_ON_SEG)) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1767 dma->dp_sglinfo.si_max_addr = attr->dma_attr_seg;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1768 dma->dp_sglinfo.si_flags &= ~_DDI_DMA_BOUNCE_ON_SEG;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1769 } else
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1770 dma->dp_sglinfo.si_max_addr = attr->dma_attr_addr_hi;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1771
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1772 hp->dmai_minxfer = attr->dma_attr_minxfer;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1773 hp->dmai_burstsizes = attr->dma_attr_burstsizes;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1774 hp->dmai_rdip = rdip;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1775 hp->dmai_attr = *attr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1776
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1777 if (attr->dma_attr_seg >= dma->dp_sglinfo.si_max_addr)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1778 dma->dp_sglinfo.si_cancross = B_FALSE;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1779 else
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1780 dma->dp_sglinfo.si_cancross = B_TRUE;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1781
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1782 /* we don't need to worry about the SPL since we do a tryenter */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1783 mutex_init(&dma->dp_mutex, NULL, MUTEX_DRIVER, NULL);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1784
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1785 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1786 * Figure out our maximum segment size. If the segment size is greater
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1787 * than 4G, we will limit it to (4G - 1) since the max size of a dma
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1788 * object (ddi_dma_obj_t.dmao_size) is 32 bits. dma_attr_seg and
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1789 * dma_attr_count_max are size-1 type values.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1790 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1791 * Maximum segment size is the largest physically contiguous chunk of
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1792 * memory that we can return from a bind (i.e. the maximum size of a
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1793 * single cookie).
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1794 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1795
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1796 /* handle the rollover cases */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1797 seg = attr->dma_attr_seg + 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1798 if (seg < attr->dma_attr_seg) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1799 seg = attr->dma_attr_seg;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1800 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1801 count_max = attr->dma_attr_count_max + 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1802 if (count_max < attr->dma_attr_count_max) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1803 count_max = attr->dma_attr_count_max;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1804 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1805
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1806 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1807 * granularity may or may not be a power of two. If it isn't, we can't
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1808 * use a simple mask.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1809 */
14707 bcf4bfbd1b70 5255 uts shouldn't open-code ISP2 Josef 'Jeff' Sipek <josef.sipek@nexenta.com> parents: 13636 diff changeset	1810 if (!ISP2(attr->dma_attr_granular)) {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1811 dma->dp_granularity_power_2 = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1812 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1813 dma->dp_granularity_power_2 = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1814 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1815
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1816 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1817 * maxxfer should be a whole multiple of granularity. If we're going to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1818 * break up a window because we're greater than maxxfer, we might as
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1819 * well make sure it's maxxfer is a whole multiple so we don't have to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1820 * worry about triming the window later on for this case.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1821 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1822 if (attr->dma_attr_granular > 1) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1823 if (dma->dp_granularity_power_2) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1824 dma->dp_maxxfer = attr->dma_attr_maxxfer -
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1825 (attr->dma_attr_maxxfer &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1826 (attr->dma_attr_granular - 1));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1827 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1828 dma->dp_maxxfer = attr->dma_attr_maxxfer -
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1829 (attr->dma_attr_maxxfer % attr->dma_attr_granular);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1830 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1831 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1832 dma->dp_maxxfer = attr->dma_attr_maxxfer;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1833 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1834
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1835 maxsegmentsize_ll = MIN(seg, dma->dp_maxxfer);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1836 maxsegmentsize_ll = MIN(maxsegmentsize_ll, count_max);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1837 if (maxsegmentsize_ll == 0 \|\| (maxsegmentsize_ll > 0xFFFFFFFF)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1838 maxsegmentsize = 0xFFFFFFFF;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1839 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1840 maxsegmentsize = maxsegmentsize_ll;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1841 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1842 dma->dp_sglinfo.si_max_cookie_size = maxsegmentsize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1843 dma->dp_sglinfo.si_segmask = attr->dma_attr_seg;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1844
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1845 /* check the ddi_dma_attr arg to make sure it makes a little sense */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1846 if (rootnex_alloc_check_parms) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1847 e = rootnex_valid_alloc_parms(attr, maxsegmentsize);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1848 if (e != DDI_SUCCESS) {
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	1849 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_ALLOC_FAIL]);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1850 (void) rootnex_dma_freehdl(dip, rdip,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1851 (ddi_dma_handle_t)hp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1852 return (e);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1853 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1854 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1855
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1856 *handlep = (ddi_dma_handle_t)hp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1857
10902 f2a954d6028e 6889384 x86 rootnex bottleneck maintaining perf counters with atomics Mark Johnson <Mark.Johnson@Sun.COM> parents: 10426 diff changeset	1858 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_ACTIVE_HDLS]);
f2a954d6028e 6889384 x86 rootnex bottleneck maintaining perf counters with atomics Mark Johnson <Mark.Johnson@Sun.COM> parents: 10426 diff changeset	1859 ROOTNEX_DPROBE1(rootnex__alloc__handle, uint64_t,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1860 rootnex_cnt[ROOTNEX_CNT_ACTIVE_HDLS]);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1861
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1862 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1863 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1864
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1865
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1866 /*
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1867 * rootnex_dma_allochdl()
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1868 * called from ddi_dma_alloc_handle().
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1869 */
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1870 static int
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1871 rootnex_dma_allochdl(dev_info_t dip, dev_info_t rdip, ddi_dma_attr_t *attr,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1872 int (waitfp)(caddr_t), caddr_t arg, ddi_dma_handle_t handlep)
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1873 {
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1874 int retval = DDI_SUCCESS;
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	1875 #if defined(__amd64) && !defined(__xpv)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1876
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1877 if (IOMMU_UNITIALIZED(rdip)) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1878 retval = iommulib_nex_open(dip, rdip);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1879
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1880 if (retval != DDI_SUCCESS && retval != DDI_ENOTSUP)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1881 return (retval);
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1882 }
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1883
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1884 if (IOMMU_UNUSED(rdip)) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1885 retval = rootnex_coredma_allochdl(dip, rdip, attr, waitfp, arg,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1886 handlep);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1887 } else {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1888 retval = iommulib_nexdma_allochdl(dip, rdip, attr,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1889 waitfp, arg, handlep);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1890 }
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1891 #else
12027 6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	1892 retval = rootnex_coredma_allochdl(dip, rdip, attr, waitfp, arg,
6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	1893 handlep);
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1894 #endif
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1895 switch (retval) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1896 case DDI_DMA_NORESOURCES:
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1897 if (waitfp != DDI_DMA_DONTWAIT) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1898 ddi_set_callback(waitfp, arg,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1899 &rootnex_state->r_dvma_call_list_id);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1900 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1901 break;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1902 case DDI_SUCCESS:
12027 6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	1903 ndi_fmc_insert(rdip, DMA_HANDLE, *handlep, NULL);
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1904 break;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1905 default:
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1906 break;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1907 }
12027 6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	1908 return (retval);
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1909 }
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1910
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1911 /ARGSUSED/
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1912 static int
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1913 rootnex_coredma_freehdl(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1914 ddi_dma_handle_t handle)
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1915 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1916 ddi_dma_impl_t *hp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1917 rootnex_dma_t *dma;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1918
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1919
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1920 hp = (ddi_dma_impl_t *)handle;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1921 dma = (rootnex_dma_t *)hp->dmai_private;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1922
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1923 /* unbind should have been called first */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1924 ASSERT(!dma->dp_inuse);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1925
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1926 mutex_destroy(&dma->dp_mutex);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1927 kmem_cache_free(rootnex_state->r_dmahdl_cache, hp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1928
10902 f2a954d6028e 6889384 x86 rootnex bottleneck maintaining perf counters with atomics Mark Johnson <Mark.Johnson@Sun.COM> parents: 10426 diff changeset	1929 ROOTNEX_DPROF_DEC(&rootnex_cnt[ROOTNEX_CNT_ACTIVE_HDLS]);
f2a954d6028e 6889384 x86 rootnex bottleneck maintaining perf counters with atomics Mark Johnson <Mark.Johnson@Sun.COM> parents: 10426 diff changeset	1930 ROOTNEX_DPROBE1(rootnex__free__handle, uint64_t,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1931 rootnex_cnt[ROOTNEX_CNT_ACTIVE_HDLS]);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1932
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1933 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1934 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1935
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1936 /*
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1937 * rootnex_dma_freehdl()
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1938 * called from ddi_dma_free_handle().
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1939 */
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1940 static int
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1941 rootnex_dma_freehdl(dev_info_t dip, dev_info_t rdip, ddi_dma_handle_t handle)
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1942 {
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1943 int ret;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1944
12027 6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	1945 ndi_fmc_remove(rdip, DMA_HANDLE, handle);
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	1946 #if defined(__amd64) && !defined(__xpv)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1947 if (IOMMU_USED(rdip))
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1948 ret = iommulib_nexdma_freehdl(dip, rdip, handle);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1949 else
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1950 #endif
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1951 ret = rootnex_coredma_freehdl(dip, rdip, handle);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1952
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1953 if (rootnex_state->r_dvma_call_list_id)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1954 ddi_run_callback(&rootnex_state->r_dvma_call_list_id);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1955
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1956 return (ret);
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1957 }
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1958
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1959 /ARGSUSED/
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1960 static int
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1961 rootnex_coredma_bindhdl(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1962 ddi_dma_handle_t handle, struct ddi_dma_req *dmareq,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	1963 ddi_dma_cookie_t cookiep, uint_t ccountp)
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	1964 {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1965 rootnex_sglinfo_t *sinfo;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1966 ddi_dma_obj_t *dmao;
13065 9377d65d657e 6955196 Intel IOMMU code should use higher-level abstraction interface (fix lint) Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 13062 diff changeset	1967 #if defined(__amd64) && !defined(__xpv)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1968 struct dvmaseg *dvs;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1969 ddi_dma_cookie_t *cookie;
13065 9377d65d657e 6955196 Intel IOMMU code should use higher-level abstraction interface (fix lint) Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 13062 diff changeset	1970 #endif
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1971 ddi_dma_attr_t *attr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1972 ddi_dma_impl_t *hp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1973 rootnex_dma_t *dma;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1974 int kmflag;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1975 int e;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	1976 uint_t ncookies;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1977
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1978 hp = (ddi_dma_impl_t *)handle;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1979 dma = (rootnex_dma_t *)hp->dmai_private;
13065 9377d65d657e 6955196 Intel IOMMU code should use higher-level abstraction interface (fix lint) Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 13062 diff changeset	1980 dmao = &dma->dp_dma;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1981 sinfo = &dma->dp_sglinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1982 attr = &hp->dmai_attr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1983
13082 87f89233b883 6964154 Missing unlock in set_all_zone_usr_proc_sys() Ethindra Ramamurthy <Ethindra.Ramamurthy@Sun.COM> parents: 13065 diff changeset	1984 /* convert the sleep flags */
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	1985 if (dmareq->dmar_fp == DDI_DMA_SLEEP) {
13082 87f89233b883 6964154 Missing unlock in set_all_zone_usr_proc_sys() Ethindra Ramamurthy <Ethindra.Ramamurthy@Sun.COM> parents: 13065 diff changeset	1986 dma->dp_sleep_flags = kmflag = KM_SLEEP;
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	1987 } else {
13082 87f89233b883 6964154 Missing unlock in set_all_zone_usr_proc_sys() Ethindra Ramamurthy <Ethindra.Ramamurthy@Sun.COM> parents: 13065 diff changeset	1988 dma->dp_sleep_flags = kmflag = KM_NOSLEEP;
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	1989 }
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	1990
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1991 hp->dmai_rflags = dmareq->dmar_flags & DMP_DDIFLAGS;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1992
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1993 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1994 * This is useful for debugging a driver. Not as useful in a production
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1995 * system. The only time this will fail is if you have a driver bug.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1996 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1997 if (rootnex_bind_check_inuse) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1998 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	1999 * No one else should ever have this lock unless someone else
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2000 * is trying to use this handle. So contention on the lock
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2001 * is the same as inuse being set.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2002 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2003 e = mutex_tryenter(&dma->dp_mutex);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2004 if (e == 0) {
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	2005 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_BIND_FAIL]);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2006 return (DDI_DMA_INUSE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2007 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2008 if (dma->dp_inuse) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2009 mutex_exit(&dma->dp_mutex);
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	2010 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_BIND_FAIL]);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2011 return (DDI_DMA_INUSE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2012 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2013 dma->dp_inuse = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2014 mutex_exit(&dma->dp_mutex);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2015 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2016
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2017 /* check the ddi_dma_attr arg to make sure it makes a little sense */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2018 if (rootnex_bind_check_parms) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2019 e = rootnex_valid_bind_parms(dmareq, attr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2020 if (e != DDI_SUCCESS) {
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	2021 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_BIND_FAIL]);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2022 rootnex_clean_dmahdl(hp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2023 return (e);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2024 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2025 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2026
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2027 /* save away the original bind info */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2028 dma->dp_dma = dmareq->dmar_object;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2029
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2030 #if defined(__amd64) && !defined(__xpv)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2031 if (IOMMU_USED(rdip)) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2032 dmao = &dma->dp_dvma;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2033 e = iommulib_nexdma_mapobject(dip, rdip, handle, dmareq, dmao);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2034 switch (e) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2035 case DDI_SUCCESS:
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2036 if (sinfo->si_cancross \|\|
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2037 dmao->dmao_obj.dvma_obj.dv_nseg != 1 \|\|
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2038 dmao->dmao_size > sinfo->si_max_cookie_size) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2039 dma->dp_dvma_used = B_TRUE;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2040 break;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2041 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2042 sinfo->si_sgl_size = 1;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2043 hp->dmai_rflags \|= DMP_NOSYNC;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2044
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2045 dma->dp_dvma_used = B_TRUE;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2046 dma->dp_need_to_free_cookie = B_FALSE;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2047
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2048 dvs = &dmao->dmao_obj.dvma_obj.dv_seg[0];
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2049 cookie = hp->dmai_cookie = dma->dp_cookies =
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2050 (ddi_dma_cookie_t *)dma->dp_prealloc_buffer;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2051 cookie->dmac_laddress = dvs->dvs_start +
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2052 dmao->dmao_obj.dvma_obj.dv_off;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2053 cookie->dmac_size = dvs->dvs_len;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2054 cookie->dmac_type = 0;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2055
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2056 ROOTNEX_DPROBE1(rootnex__bind__dvmafast, dev_info_t *,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2057 rdip);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2058 goto fast;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2059 case DDI_ENOTSUP:
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2060 break;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2061 default:
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2062 rootnex_clean_dmahdl(hp);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2063 return (e);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2064 }
7589 7de800909a06 PSARC 2008/560 Intel IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7542 diff changeset	2065 }
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2066 #endif
7589 7de800909a06 PSARC 2008/560 Intel IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7542 diff changeset	2067
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2068 /*
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2069 * Figure out a rough estimate of what maximum number of pages
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2070 * this buffer could use (a high estimate of course).
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2071 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2072 sinfo->si_max_pages = mmu_btopr(dma->dp_dma.dmao_size) + 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2073
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2074 if (dma->dp_dvma_used) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2075 /*
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2076 * The number of physical pages is the worst case.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2077 *
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2078 * For DVMA, the worst case is the length divided
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2079 * by the maximum cookie length, plus 1. Add to that
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2080 * the number of segment boundaries potentially crossed, and
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2081 * the additional number of DVMA segments that was returned.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2082 *
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2083 * In the normal case, for modern devices, si_cancross will
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2084 * be false, and dv_nseg will be 1, and the fast path will
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2085 * have been taken above.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2086 */
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2087 ncookies = (dma->dp_dma.dmao_size / sinfo->si_max_cookie_size)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2088 + 1;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2089 if (sinfo->si_cancross)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2090 ncookies +=
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2091 (dma->dp_dma.dmao_size / attr->dma_attr_seg) + 1;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2092 ncookies += (dmao->dmao_obj.dvma_obj.dv_nseg - 1);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2093
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2094 sinfo->si_max_pages = MIN(sinfo->si_max_pages, ncookies);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2095 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2096
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2097 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2098 * We'll use the pre-allocated cookies for any bind that will always
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2099 * fit (more important to be consistent, we don't want to create
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2100 * additional degenerate cases).
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2101 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2102 if (sinfo->si_max_pages <= rootnex_state->r_prealloc_cookies) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2103 dma->dp_cookies = (ddi_dma_cookie_t *)dma->dp_prealloc_buffer;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2104 dma->dp_need_to_free_cookie = B_FALSE;
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	2105 ROOTNEX_DPROBE2(rootnex__bind__prealloc, dev_info_t *, rdip,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2106 uint_t, sinfo->si_max_pages);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2107
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2108 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2109 * For anything larger than that, we'll go ahead and allocate the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2110 * maximum number of pages we expect to see. Hopefuly, we won't be
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2111 * seeing this path in the fast path for high performance devices very
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2112 * frequently.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2113 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2114 * a ddi bind interface that allowed the driver to provide storage to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2115 * the bind interface would speed this case up.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2116 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2117 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2118 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2119 * Save away how much memory we allocated. If we're doing a
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2120 * nosleep, the alloc could fail...
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2121 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2122 dma->dp_cookie_size = sinfo->si_max_pages *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2123 sizeof (ddi_dma_cookie_t);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2124 dma->dp_cookies = kmem_alloc(dma->dp_cookie_size, kmflag);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2125 if (dma->dp_cookies == NULL) {
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	2126 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_BIND_FAIL]);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2127 rootnex_clean_dmahdl(hp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2128 return (DDI_DMA_NORESOURCES);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2129 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2130 dma->dp_need_to_free_cookie = B_TRUE;
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	2131 ROOTNEX_DPROBE2(rootnex__bind__alloc, dev_info_t *, rdip,
331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	2132 uint_t, sinfo->si_max_pages);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2133 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2134 hp->dmai_cookie = dma->dp_cookies;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2135
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2136 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2137 * Get the real sgl. rootnex_get_sgl will fill in cookie array while
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2138 * looking at the constraints in the dma structure. It will then put
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2139 * some additional state about the sgl in the dma struct (i.e. is
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2140 * the sgl clean, or do we need to do some munging; how many pages
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2141 * need to be copied, etc.)
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2142 */
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2143 if (dma->dp_dvma_used)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2144 rootnex_dvma_get_sgl(dmao, dma->dp_cookies, &dma->dp_sglinfo);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2145 else
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2146 rootnex_get_sgl(dmao, dma->dp_cookies, &dma->dp_sglinfo);
7589 7de800909a06 PSARC 2008/560 Intel IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7542 diff changeset	2147
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2148 out:
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2149 ASSERT(sinfo->si_sgl_size <= sinfo->si_max_pages);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2150 /* if we don't need a copy buffer, we don't need to sync */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2151 if (sinfo->si_copybuf_req == 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2152 hp->dmai_rflags \|= DMP_NOSYNC;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2153 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2154
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2155 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2156 * if we don't need the copybuf and we don't need to do a partial, we
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2157 * hit the fast path. All the high performance devices should be trying
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2158 * to hit this path. To hit this path, a device should be able to reach
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2159 * all of memory, shouldn't try to bind more than it can transfer, and
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2160 * the buffer shouldn't require more cookies than the driver/device can
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2161 * handle [sgllen]).
13364 b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	2162 *
b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	2163 * Note that negative values of dma_attr_sgllen are supposed
b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	2164 * to mean unlimited, but we just cast them to mean a
b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	2165 * "ridiculous large limit". This saves some extra checks on
b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	2166 * hot paths.
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2167 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2168 if ((sinfo->si_copybuf_req == 0) &&
13364 b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	2169 (sinfo->si_sgl_size <= (unsigned)attr->dma_attr_sgllen) &&
16725 1a38f0e5c9d4 8624 xhci and nvme can't bind DMA memory with IOMMU enabled Hans Rosenfeld <hans.rosenfeld@joyent.com> parents: 15433 diff changeset	2170 (dmao->dmao_size <= dma->dp_maxxfer)) {
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2171 fast:
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2172 /*
5591 2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2173 * If the driver supports FMA, insert the handle in the FMA DMA
2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2174 * handle cache.
2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2175 */
12027 6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	2176 if (attr->dma_attr_flags & DDI_DMA_FLAGERR)
5591 2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2177 hp->dmai_error.err_cf = rootnex_dma_check;
2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2178
2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2179 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2180 * copy out the first cookie and ccountp, set the cookie
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2181 * pointer to the second cookie. The first cookie is passed
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2182 * back on the stack. Additional cookies are accessed via
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2183 * ddi_dma_nextcookie()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2184 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2185 *cookiep = dma->dp_cookies[0];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2186 *ccountp = sinfo->si_sgl_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2187 hp->dmai_cookie++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2188 hp->dmai_rflags &= ~DDI_DMA_PARTIAL;
19431 5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	2189 hp->dmai_ncookies = *ccountp;
5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	2190 hp->dmai_curcookie = 1;
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	2191 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_ACTIVE_BINDS]);
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2192 ROOTNEX_DPROBE4(rootnex__bind__fast, dev_info_t *, rdip,
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2193 uint64_t, rootnex_cnt[ROOTNEX_CNT_ACTIVE_BINDS],
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2194 uint_t, dmao->dmao_size, uint_t, *ccountp);
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2195
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2196
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2197 return (DDI_DMA_MAPPED);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2198 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2199
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2200 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2201 * go to the slow path, we may need to alloc more memory, create
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2202 * multiple windows, and munge up a sgl to make the device happy.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2203 */
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2204
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2205 /*
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2206 * With the IOMMU mapobject method used, we should never hit
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2207 * the slow path. If we do, something is seriously wrong.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2208 * Clean up and return an error.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2209 */
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2210
13216 8bfa85c99f86 185 PV boot panics due to unresolved iommulib symbol Albert Lee <trisk@opensolaris.org> parents: 13082 diff changeset	2211 #if defined(__amd64) && !defined(__xpv)
8bfa85c99f86 185 PV boot panics due to unresolved iommulib symbol Albert Lee <trisk@opensolaris.org> parents: 13082 diff changeset	2212
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2213 if (dma->dp_dvma_used) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2214 (void) iommulib_nexdma_unmapobject(dip, rdip, handle,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2215 &dma->dp_dvma);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2216 e = DDI_DMA_NOMAPPING;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2217 } else {
13216 8bfa85c99f86 185 PV boot panics due to unresolved iommulib symbol Albert Lee <trisk@opensolaris.org> parents: 13082 diff changeset	2218 #endif
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2219 e = rootnex_bind_slowpath(hp, dmareq, dma, attr, &dma->dp_dma,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2220 kmflag);
13216 8bfa85c99f86 185 PV boot panics due to unresolved iommulib symbol Albert Lee <trisk@opensolaris.org> parents: 13082 diff changeset	2221 #if defined(__amd64) && !defined(__xpv)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2222 }
13216 8bfa85c99f86 185 PV boot panics due to unresolved iommulib symbol Albert Lee <trisk@opensolaris.org> parents: 13082 diff changeset	2223 #endif
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2224 if ((e != DDI_DMA_MAPPED) && (e != DDI_DMA_PARTIAL_MAP)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2225 if (dma->dp_need_to_free_cookie) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2226 kmem_free(dma->dp_cookies, dma->dp_cookie_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2227 }
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	2228 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_BIND_FAIL]);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2229 rootnex_clean_dmahdl(hp); /* must be after free cookie */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2230 return (e);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2231 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2232
5591 2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2233 /*
2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2234 * If the driver supports FMA, insert the handle in the FMA DMA handle
2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2235 * cache.
2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2236 */
12027 6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	2237 if (attr->dma_attr_flags & DDI_DMA_FLAGERR)
5591 2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2238 hp->dmai_error.err_cf = rootnex_dma_check;
2526bceec9d3 6617132 panic in fmc_grow w/ IDR IDR127941-01 stephh parents: 5251 diff changeset	2239
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2240 /* if the first window uses the copy buffer, sync it for the device */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2241 if ((dma->dp_window[dma->dp_current_win].wd_dosync) &&
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2242 (hp->dmai_rflags & DDI_DMA_WRITE)) {
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2243 (void) rootnex_coredma_sync(dip, rdip, handle, 0, 0,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2244 DDI_DMA_SYNC_FORDEV);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2245 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2246
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2247 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2248 * copy out the first cookie and ccountp, set the cookie pointer to the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2249 * second cookie. Make sure the partial flag is set/cleared correctly.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2250 * If we have a partial map (i.e. multiple windows), the number of
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2251 * cookies we return is the number of cookies in the first window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2252 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2253 if (e == DDI_DMA_MAPPED) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2254 hp->dmai_rflags &= ~DDI_DMA_PARTIAL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2255 *ccountp = sinfo->si_sgl_size;
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2256 hp->dmai_nwin = 1;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2257 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2258 hp->dmai_rflags \|= DDI_DMA_PARTIAL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2259 *ccountp = dma->dp_window[dma->dp_current_win].wd_cookie_cnt;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2260 ASSERT(hp->dmai_nwin <= dma->dp_max_win);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2261 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2262 *cookiep = dma->dp_cookies[0];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2263 hp->dmai_cookie++;
19431 5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	2264 hp->dmai_ncookies = *ccountp;
5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	2265 hp->dmai_curcookie = 1;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2266
10902 f2a954d6028e 6889384 x86 rootnex bottleneck maintaining perf counters with atomics Mark Johnson <Mark.Johnson@Sun.COM> parents: 10426 diff changeset	2267 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_ACTIVE_BINDS]);
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2268 ROOTNEX_DPROBE4(rootnex__bind__slow, dev_info_t *, rdip, uint64_t,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2269 rootnex_cnt[ROOTNEX_CNT_ACTIVE_BINDS], uint_t,
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2270 dmao->dmao_size, uint_t, *ccountp);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2271 return (e);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2272 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2273
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2274 /*
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2275 * rootnex_dma_bindhdl()
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2276 * called from ddi_dma_addr_bind_handle() and ddi_dma_buf_bind_handle().
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2277 */
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2278 static int
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2279 rootnex_dma_bindhdl(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2280 ddi_dma_handle_t handle, struct ddi_dma_req *dmareq,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2281 ddi_dma_cookie_t cookiep, uint_t ccountp)
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2282 {
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2283 int ret;
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2284 #if defined(__amd64) && !defined(__xpv)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2285 if (IOMMU_USED(rdip))
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2286 ret = iommulib_nexdma_bindhdl(dip, rdip, handle, dmareq,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2287 cookiep, ccountp);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2288 else
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2289 #endif
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2290 ret = rootnex_coredma_bindhdl(dip, rdip, handle, dmareq,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2291 cookiep, ccountp);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2292
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2293 if (ret == DDI_DMA_NORESOURCES && dmareq->dmar_fp != DDI_DMA_DONTWAIT) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2294 ddi_set_callback(dmareq->dmar_fp, dmareq->dmar_arg,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2295 &rootnex_state->r_dvma_call_list_id);
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2296 }
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2297
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2298 return (ret);
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2299 }
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2300
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2301
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2302
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2303 /ARGSUSED/
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2304 static int
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2305 rootnex_coredma_unbindhdl(dev_info_t dip, dev_info_t rdip,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2306 ddi_dma_handle_t handle)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2307 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2308 ddi_dma_impl_t *hp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2309 rootnex_dma_t *dma;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2310 int e;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2311
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2312 hp = (ddi_dma_impl_t *)handle;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2313 dma = (rootnex_dma_t *)hp->dmai_private;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2314
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2315 /* make sure the buffer wasn't free'd before calling unbind */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2316 if (rootnex_unbind_verify_buffer) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2317 e = rootnex_verify_buffer(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2318 if (e != DDI_SUCCESS) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2319 ASSERT(0);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2320 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2321 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2322 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2323
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2324 /* sync the current window before unbinding the buffer */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2325 if (dma->dp_window && dma->dp_window[dma->dp_current_win].wd_dosync &&
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2326 (hp->dmai_rflags & DDI_DMA_READ)) {
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2327 (void) rootnex_coredma_sync(dip, rdip, handle, 0, 0,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2328 DDI_DMA_SYNC_FORCPU);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2329 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2330
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2331 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2332 * cleanup and copy buffer or window state. if we didn't use the copy
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2333 * buffer or windows, there won't be much to do :-)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2334 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2335 rootnex_teardown_copybuf(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2336 rootnex_teardown_windows(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2337
13216 8bfa85c99f86 185 PV boot panics due to unresolved iommulib symbol Albert Lee <trisk@opensolaris.org> parents: 13082 diff changeset	2338 #if defined(__amd64) && !defined(__xpv)
16727 e55d9e3ffb7b 8622 panic in PTE_set_all() Hans Rosenfeld <hans.rosenfeld@joyent.com> parents: 16725 diff changeset	2339 if (IOMMU_USED(rdip) && dma->dp_dvma_used)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2340 (void) iommulib_nexdma_unmapobject(dip, rdip, handle,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2341 &dma->dp_dvma);
13216 8bfa85c99f86 185 PV boot panics due to unresolved iommulib symbol Albert Lee <trisk@opensolaris.org> parents: 13082 diff changeset	2342 #endif
7589 7de800909a06 PSARC 2008/560 Intel IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7542 diff changeset	2343
7de800909a06 PSARC 2008/560 Intel IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7542 diff changeset	2344 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2345 * If we had to allocate space to for the worse case sgl (it didn't
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2346 * fit into our pre-allocate buffer), free that up now
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2347 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2348 if (dma->dp_need_to_free_cookie) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2349 kmem_free(dma->dp_cookies, dma->dp_cookie_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2350 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2351
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2352 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2353 * clean up the handle so it's ready for the next bind (i.e. if the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2354 * handle is reused).
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2355 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2356 rootnex_clean_dmahdl(hp);
12027 6c3efe08f39d 6896082 Hot codepath seen in FMA hurts TX throughput of FMA capable NIC drivers Stephen Hanson <Stephen.Hanson@Sun.COM> parents: 11793 diff changeset	2357 hp->dmai_error.err_cf = NULL;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2358
10902 f2a954d6028e 6889384 x86 rootnex bottleneck maintaining perf counters with atomics Mark Johnson <Mark.Johnson@Sun.COM> parents: 10426 diff changeset	2359 ROOTNEX_DPROF_DEC(&rootnex_cnt[ROOTNEX_CNT_ACTIVE_BINDS]);
f2a954d6028e 6889384 x86 rootnex bottleneck maintaining perf counters with atomics Mark Johnson <Mark.Johnson@Sun.COM> parents: 10426 diff changeset	2360 ROOTNEX_DPROBE1(rootnex__unbind, uint64_t,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2361 rootnex_cnt[ROOTNEX_CNT_ACTIVE_BINDS]);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2362
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2363 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2364 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2365
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2366 /*
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2367 * rootnex_dma_unbindhdl()
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2368 * called from ddi_dma_unbind_handle()
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2369 */
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2370 /ARGSUSED/
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2371 static int
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2372 rootnex_dma_unbindhdl(dev_info_t dip, dev_info_t rdip,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2373 ddi_dma_handle_t handle)
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2374 {
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2375 int ret;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2376
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2377 #if defined(__amd64) && !defined(__xpv)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2378 if (IOMMU_USED(rdip))
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2379 ret = iommulib_nexdma_unbindhdl(dip, rdip, handle);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2380 else
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2381 #endif
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2382 ret = rootnex_coredma_unbindhdl(dip, rdip, handle);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2383
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2384 if (rootnex_state->r_dvma_call_list_id)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2385 ddi_run_callback(&rootnex_state->r_dvma_call_list_id);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2386
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2387 return (ret);
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2388 }
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2389
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	2390 #if defined(__amd64) && !defined(__xpv)
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2391
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2392 static int
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2393 rootnex_coredma_get_sleep_flags(ddi_dma_handle_t handle)
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2394 {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2395 ddi_dma_impl_t hp = (ddi_dma_impl_t )handle;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2396 rootnex_dma_t dma = (rootnex_dma_t )hp->dmai_private;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2397
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2398 if (dma->dp_sleep_flags != KM_SLEEP &&
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2399 dma->dp_sleep_flags != KM_NOSLEEP)
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2400 cmn_err(CE_PANIC, "kmem sleep flags not set in DMA handle");
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2401 return (dma->dp_sleep_flags);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2402 }
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2403 /ARGSUSED/
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2404 static void
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2405 rootnex_coredma_reset_cookies(dev_info_t *dip, ddi_dma_handle_t handle)
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2406 {
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2407 ddi_dma_impl_t hp = (ddi_dma_impl_t )handle;
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2408 rootnex_dma_t dma = (rootnex_dma_t )hp->dmai_private;
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2409 rootnex_window_t *window;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2410
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2411 if (dma->dp_window) {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2412 window = &dma->dp_window[dma->dp_current_win];
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2413 hp->dmai_cookie = window->wd_first_cookie;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2414 } else {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2415 hp->dmai_cookie = dma->dp_cookies;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2416 }
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2417 hp->dmai_cookie++;
19431 5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	2418 hp->dmai_curcookie = 1;
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2419 }
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2420
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2421 /ARGSUSED/
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2422 static int
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2423 rootnex_coredma_get_cookies(dev_info_t *dip, ddi_dma_handle_t handle,
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2424 ddi_dma_cookie_t *cookiepp, uint_t ccountp)
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2425 {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2426 int i;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2427 int km_flags;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2428 ddi_dma_impl_t hp = (ddi_dma_impl_t )handle;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2429 rootnex_dma_t dma = (rootnex_dma_t )hp->dmai_private;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2430 rootnex_window_t *window;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2431 ddi_dma_cookie_t *cp;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2432 ddi_dma_cookie_t *cookie;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2433
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2434 ASSERT(*cookiepp == NULL);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2435 ASSERT(*ccountp == 0);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2436
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2437 if (dma->dp_window) {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2438 window = &dma->dp_window[dma->dp_current_win];
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2439 cp = window->wd_first_cookie;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2440 *ccountp = window->wd_cookie_cnt;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2441 } else {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2442 cp = dma->dp_cookies;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2443 *ccountp = dma->dp_sglinfo.si_sgl_size;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2444 }
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2445
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2446 km_flags = rootnex_coredma_get_sleep_flags(handle);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2447 cookie = kmem_zalloc(sizeof (ddi_dma_cookie_t) * (*ccountp), km_flags);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2448 if (cookie == NULL) {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2449 return (DDI_DMA_NORESOURCES);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2450 }
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2451
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2452 for (i = 0; i < *ccountp; i++) {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2453 cookie[i].dmac_notused = cp[i].dmac_notused;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2454 cookie[i].dmac_type = cp[i].dmac_type;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2455 cookie[i].dmac_address = cp[i].dmac_address;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2456 cookie[i].dmac_size = cp[i].dmac_size;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2457 }
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2458
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2459 *cookiepp = cookie;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2460
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2461 return (DDI_SUCCESS);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2462 }
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2463
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2464 /ARGSUSED/
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2465 static int
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2466 rootnex_coredma_set_cookies(dev_info_t *dip, ddi_dma_handle_t handle,
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2467 ddi_dma_cookie_t *cookiep, uint_t ccount)
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2468 {
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2469 ddi_dma_impl_t hp = (ddi_dma_impl_t )handle;
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2470 rootnex_dma_t dma = (rootnex_dma_t )hp->dmai_private;
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2471 rootnex_window_t *window;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2472 ddi_dma_cookie_t *cur_cookiep;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2473
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2474 ASSERT(cookiep);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2475 ASSERT(ccount != 0);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2476 ASSERT(dma->dp_need_to_switch_cookies == B_FALSE);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2477
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2478 if (dma->dp_window) {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2479 window = &dma->dp_window[dma->dp_current_win];
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2480 dma->dp_saved_cookies = window->wd_first_cookie;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2481 window->wd_first_cookie = cookiep;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2482 ASSERT(ccount == window->wd_cookie_cnt);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2483 cur_cookiep = (hp->dmai_cookie - dma->dp_saved_cookies)
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2484 + window->wd_first_cookie;
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2485 } else {
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2486 dma->dp_saved_cookies = dma->dp_cookies;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2487 dma->dp_cookies = cookiep;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2488 ASSERT(ccount == dma->dp_sglinfo.si_sgl_size);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2489 cur_cookiep = (hp->dmai_cookie - dma->dp_saved_cookies)
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2490 + dma->dp_cookies;
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2491 }
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2492
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2493 dma->dp_need_to_switch_cookies = B_TRUE;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2494 hp->dmai_cookie = cur_cookiep;
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2495
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2496 return (DDI_SUCCESS);
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	2497 }
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2498
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2499 /ARGSUSED/
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2500 static int
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2501 rootnex_coredma_clear_cookies(dev_info_t *dip, ddi_dma_handle_t handle)
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2502 {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2503 ddi_dma_impl_t hp = (ddi_dma_impl_t )handle;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2504 rootnex_dma_t dma = (rootnex_dma_t )hp->dmai_private;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2505 rootnex_window_t *window;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2506 ddi_dma_cookie_t *cur_cookiep;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2507 ddi_dma_cookie_t *cookie_array;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2508 uint_t ccount;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2509
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2510 /* check if cookies have not been switched */
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2511 if (dma->dp_need_to_switch_cookies == B_FALSE)
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2512 return (DDI_SUCCESS);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2513
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2514 ASSERT(dma->dp_saved_cookies);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2515
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2516 if (dma->dp_window) {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2517 window = &dma->dp_window[dma->dp_current_win];
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2518 cookie_array = window->wd_first_cookie;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2519 window->wd_first_cookie = dma->dp_saved_cookies;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2520 dma->dp_saved_cookies = NULL;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2521 ccount = window->wd_cookie_cnt;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2522 cur_cookiep = (hp->dmai_cookie - cookie_array)
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2523 + window->wd_first_cookie;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2524 } else {
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2525 cookie_array = dma->dp_cookies;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2526 dma->dp_cookies = dma->dp_saved_cookies;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2527 dma->dp_saved_cookies = NULL;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2528 ccount = dma->dp_sglinfo.si_sgl_size;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2529 cur_cookiep = (hp->dmai_cookie - cookie_array)
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2530 + dma->dp_cookies;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2531 }
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2532
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2533 kmem_free(cookie_array, sizeof (ddi_dma_cookie_t) * ccount);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2534
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2535 hp->dmai_cookie = cur_cookiep;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2536
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2537 dma->dp_need_to_switch_cookies = B_FALSE;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2538
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2539 return (DDI_SUCCESS);
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2540 }
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2541
7617 1e2a36b96537 6748924 fix lint warnings in putback for 6747701 Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7613 diff changeset	2542 #endif
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2543
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2544 static struct as *
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2545 rootnex_get_as(ddi_dma_obj_t *dmao)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2546 {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2547 struct as *asp;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2548
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2549 switch (dmao->dmao_type) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2550 case DMA_OTYP_VADDR:
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2551 case DMA_OTYP_BUFVADDR:
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2552 asp = dmao->dmao_obj.virt_obj.v_as;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2553 if (asp == NULL)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2554 asp = &kas;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2555 break;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2556 default:
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2557 asp = NULL;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2558 break;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2559 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2560 return (asp);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2561 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2562
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2563 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2564 * rootnex_verify_buffer()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2565 * verify buffer wasn't free'd
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2566 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2567 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2568 rootnex_verify_buffer(rootnex_dma_t *dma)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2569 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2570 page_t **pplist;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2571 caddr_t vaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2572 uint_t pcnt;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2573 uint_t poff;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2574 page_t *pp;
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	2575 char b;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2576 int i;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2577
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2578 /* Figure out how many pages this buffer occupies */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2579 if (dma->dp_dma.dmao_type == DMA_OTYP_PAGES) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2580 poff = dma->dp_dma.dmao_obj.pp_obj.pp_offset & MMU_PAGEOFFSET;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2581 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2582 vaddr = dma->dp_dma.dmao_obj.virt_obj.v_addr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2583 poff = (uintptr_t)vaddr & MMU_PAGEOFFSET;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2584 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2585 pcnt = mmu_btopr(dma->dp_dma.dmao_size + poff);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2586
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2587 switch (dma->dp_dma.dmao_type) {
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	2588 case DMA_OTYP_PAGES:
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2589 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2590 * for a linked list of pp's walk through them to make sure
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2591 * they're locked and not free.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2592 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2593 pp = dma->dp_dma.dmao_obj.pp_obj.pp_pp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2594 for (i = 0; i < pcnt; i++) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2595 if (PP_ISFREE(pp) \|\| !PAGE_LOCKED(pp)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2596 return (DDI_FAILURE);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	2597 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2598 pp = pp->p_next;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	2599 }
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	2600 break;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2601
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	2602 case DMA_OTYP_VADDR:
68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	2603 case DMA_OTYP_BUFVADDR:
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2604 pplist = dma->dp_dma.dmao_obj.virt_obj.v_priv;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2605 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2606 * for an array of pp's walk through them to make sure they're
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2607 * not free. It's possible that they may not be locked.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2608 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2609 if (pplist) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2610 for (i = 0; i < pcnt; i++) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2611 if (PP_ISFREE(pplist[i])) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2612 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2613 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2614 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2615
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2616 /* For a virtual address, try to peek at each page */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2617 } else {
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2618 if (rootnex_get_as(&dma->dp_dma) == &kas) {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2619 for (i = 0; i < pcnt; i++) {
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	2620 if (ddi_peek8(NULL, vaddr, &b) ==
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	2621 DDI_FAILURE)
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2622 return (DDI_FAILURE);
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	2623 vaddr += MMU_PAGESIZE;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2624 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2625 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2626 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2627 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2628
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2629 default:
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2630 cmn_err(CE_PANIC, "rootnex_verify_buffer: bad DMA object");
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2631 break;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2632 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2633
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2634 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2635 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2636
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2637
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2638 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2639 * rootnex_clean_dmahdl()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2640 * Clean the dma handle. This should be called on a handle alloc and an
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2641 * unbind handle. Set the handle state to the default settings.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2642 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2643 static void
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2644 rootnex_clean_dmahdl(ddi_dma_impl_t *hp)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2645 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2646 rootnex_dma_t *dma;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2647
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2648
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2649 dma = (rootnex_dma_t *)hp->dmai_private;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2650
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2651 hp->dmai_nwin = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2652 dma->dp_current_cookie = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2653 dma->dp_copybuf_size = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2654 dma->dp_window = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2655 dma->dp_cbaddr = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2656 dma->dp_inuse = B_FALSE;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2657 dma->dp_dvma_used = B_FALSE;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2658 dma->dp_need_to_free_cookie = B_FALSE;
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2659 dma->dp_need_to_switch_cookies = B_FALSE;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2660 dma->dp_saved_cookies = NULL;
dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	2661 dma->dp_sleep_flags = KM_PANIC;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2662 dma->dp_need_to_free_window = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2663 dma->dp_partial_required = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2664 dma->dp_trim_required = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2665 dma->dp_sglinfo.si_copybuf_req = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2666 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2667 dma->dp_cb_remaping = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2668 dma->dp_kva = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2669 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2670
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2671 /* FMA related initialization */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2672 hp->dmai_fault = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2673 hp->dmai_fault_check = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2674 hp->dmai_fault_notify = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2675 hp->dmai_error.err_ena = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2676 hp->dmai_error.err_status = DDI_FM_OK;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2677 hp->dmai_error.err_expected = DDI_FM_ERR_UNEXPECTED;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2678 hp->dmai_error.err_ontrap = NULL;
19431 5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	2679
5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	2680 /* Cookie tracking */
5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	2681 hp->dmai_ncookies = 0;
5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	2682 hp->dmai_curcookie = 0;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2683 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2684
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2685
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2686 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2687 * rootnex_valid_alloc_parms()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2688 * Called in ddi_dma_alloc_handle path to validate its parameters.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2689 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2690 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2691 rootnex_valid_alloc_parms(ddi_dma_attr_t *attr, uint_t maxsegmentsize)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2692 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2693 if ((attr->dma_attr_seg < MMU_PAGEOFFSET) \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2694 (attr->dma_attr_count_max < MMU_PAGEOFFSET) \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2695 (attr->dma_attr_granular > MMU_PAGESIZE) \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2696 (attr->dma_attr_maxxfer < MMU_PAGESIZE)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2697 return (DDI_DMA_BADATTR);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2698 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2699
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2700 if (attr->dma_attr_addr_hi <= attr->dma_attr_addr_lo) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2701 return (DDI_DMA_BADATTR);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2702 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2703
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2704 if ((attr->dma_attr_seg & MMU_PAGEOFFSET) != MMU_PAGEOFFSET \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2705 MMU_PAGESIZE & (attr->dma_attr_granular - 1) \|\|
13364 b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	2706 attr->dma_attr_sgllen == 0) {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2707 return (DDI_DMA_BADATTR);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2708 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2709
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2710 /* We should be able to DMA into every byte offset in a page */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2711 if (maxsegmentsize < MMU_PAGESIZE) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2712 return (DDI_DMA_BADATTR);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2713 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2714
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2715 /* if we're bouncing on seg, seg must be <= addr_hi */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2716 if ((attr->dma_attr_flags & _DDI_DMA_BOUNCE_ON_SEG) &&
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2717 (attr->dma_attr_seg > attr->dma_attr_addr_hi)) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2718 return (DDI_DMA_BADATTR);
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2719 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2720 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2721 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2722
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2723 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2724 * rootnex_valid_bind_parms()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2725 * Called in ddi_dma_*_bind_handle path to validate its parameters.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2726 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2727 /* ARGSUSED */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2728 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2729 rootnex_valid_bind_parms(ddi_dma_req_t dmareq, ddi_dma_attr_t attr)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2730 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2731 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2732 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2733 * we only support up to a 2G-1 transfer size on 32-bit kernels so
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2734 * we can track the offset for the obsoleted interfaces.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2735 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2736 if (dmareq->dmar_object.dmao_size > 0x7FFFFFFF) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2737 return (DDI_DMA_TOOBIG);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2738 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2739 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2740
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2741 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2742 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2743
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2744
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2745 /*
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2746 * rootnex_need_bounce_seg()
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2747 * check to see if the buffer lives on both side of the seg.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2748 */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2749 static boolean_t
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2750 rootnex_need_bounce_seg(ddi_dma_obj_t dmar_object, rootnex_sglinfo_t sglinfo)
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2751 {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2752 ddi_dma_atyp_t buftype;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2753 rootnex_addr_t raddr;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2754 boolean_t lower_addr;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2755 boolean_t upper_addr;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2756 uint64_t offset;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2757 page_t **pplist;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2758 uint64_t paddr;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2759 uint32_t psize;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2760 uint32_t size;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2761 caddr_t vaddr;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2762 uint_t pcnt;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2763 page_t *pp;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2764
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2765
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2766 /* shortcuts */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2767 pplist = dmar_object->dmao_obj.virt_obj.v_priv;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2768 vaddr = dmar_object->dmao_obj.virt_obj.v_addr;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2769 buftype = dmar_object->dmao_type;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2770 size = dmar_object->dmao_size;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2771
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2772 lower_addr = B_FALSE;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2773 upper_addr = B_FALSE;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2774 pcnt = 0;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2775
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2776 /*
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2777 * Process the first page to handle the initial offset of the buffer.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2778 * We'll use the base address we get later when we loop through all
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2779 * the pages.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2780 */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2781 if (buftype == DMA_OTYP_PAGES) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2782 pp = dmar_object->dmao_obj.pp_obj.pp_pp;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2783 offset = dmar_object->dmao_obj.pp_obj.pp_offset &
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2784 MMU_PAGEOFFSET;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2785 paddr = pfn_to_pa(pp->p_pagenum) + offset;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2786 psize = MIN(size, (MMU_PAGESIZE - offset));
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2787 pp = pp->p_next;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2788 sglinfo->si_asp = NULL;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2789 } else if (pplist != NULL) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2790 offset = (uintptr_t)vaddr & MMU_PAGEOFFSET;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2791 sglinfo->si_asp = dmar_object->dmao_obj.virt_obj.v_as;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2792 if (sglinfo->si_asp == NULL) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2793 sglinfo->si_asp = &kas;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2794 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2795 paddr = pfn_to_pa(pplist[pcnt]->p_pagenum);
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2796 paddr += offset;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2797 psize = MIN(size, (MMU_PAGESIZE - offset));
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2798 pcnt++;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2799 } else {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2800 offset = (uintptr_t)vaddr & MMU_PAGEOFFSET;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2801 sglinfo->si_asp = dmar_object->dmao_obj.virt_obj.v_as;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2802 if (sglinfo->si_asp == NULL) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2803 sglinfo->si_asp = &kas;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2804 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2805 paddr = pfn_to_pa(hat_getpfnum(sglinfo->si_asp->a_hat, vaddr));
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2806 paddr += offset;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2807 psize = MIN(size, (MMU_PAGESIZE - offset));
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2808 vaddr += psize;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2809 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2810
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2811 raddr = ROOTNEX_PADDR_TO_RBASE(paddr);
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2812
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2813 if ((raddr + psize) > sglinfo->si_segmask) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2814 upper_addr = B_TRUE;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2815 } else {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2816 lower_addr = B_TRUE;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2817 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2818 size -= psize;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2819
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2820 /*
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2821 * Walk through the rest of the pages in the buffer. Track to see
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2822 * if we have pages on both sides of the segment boundary.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2823 */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2824 while (size > 0) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2825 /* partial or full page */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2826 psize = MIN(size, MMU_PAGESIZE);
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2827
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2828 if (buftype == DMA_OTYP_PAGES) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2829 /* get the paddr from the page_t */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2830 ASSERT(!PP_ISFREE(pp) && PAGE_LOCKED(pp));
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2831 paddr = pfn_to_pa(pp->p_pagenum);
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2832 pp = pp->p_next;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2833 } else if (pplist != NULL) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2834 /* index into the array of page_t's to get the paddr */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2835 ASSERT(!PP_ISFREE(pplist[pcnt]));
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2836 paddr = pfn_to_pa(pplist[pcnt]->p_pagenum);
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2837 pcnt++;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2838 } else {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2839 /* call into the VM to get the paddr */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2840 paddr = pfn_to_pa(hat_getpfnum(sglinfo->si_asp->a_hat,
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2841 vaddr));
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2842 vaddr += psize;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2843 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2844
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2845 raddr = ROOTNEX_PADDR_TO_RBASE(paddr);
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2846
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2847 if ((raddr + psize) > sglinfo->si_segmask) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2848 upper_addr = B_TRUE;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2849 } else {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2850 lower_addr = B_TRUE;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2851 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2852 /*
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2853 * if the buffer lives both above and below the segment
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2854 * boundary, or the current page is the page immediately
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2855 * after the segment, we will use a copy/bounce buffer for
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2856 * all pages > seg.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2857 */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2858 if ((lower_addr && upper_addr) \|\|
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2859 (raddr == (sglinfo->si_segmask + 1))) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2860 return (B_TRUE);
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2861 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2862
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2863 size -= psize;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2864 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2865
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2866 return (B_FALSE);
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2867 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2868
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2869 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2870 * rootnex_get_sgl()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2871 * Called in bind fastpath to get the sgl. Most of this will be replaced
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2872 * with a call to the vm layer when vm2.0 comes around...
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2873 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2874 static void
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2875 rootnex_get_sgl(ddi_dma_obj_t dmar_object, ddi_dma_cookie_t sgl,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2876 rootnex_sglinfo_t *sglinfo)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2877 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2878 ddi_dma_atyp_t buftype;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	2879 rootnex_addr_t raddr;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2880 uint64_t last_page;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2881 uint64_t offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2882 uint64_t addrhi;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2883 uint64_t addrlo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2884 uint64_t maxseg;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2885 page_t **pplist;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2886 uint64_t paddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2887 uint32_t psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2888 uint32_t size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2889 caddr_t vaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2890 uint_t pcnt;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2891 page_t *pp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2892 uint_t cnt;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2893
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2894
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2895 /* shortcuts */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2896 pplist = dmar_object->dmao_obj.virt_obj.v_priv;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2897 vaddr = dmar_object->dmao_obj.virt_obj.v_addr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2898 maxseg = sglinfo->si_max_cookie_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2899 buftype = dmar_object->dmao_type;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2900 addrhi = sglinfo->si_max_addr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2901 addrlo = sglinfo->si_min_addr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2902 size = dmar_object->dmao_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2903
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2904 pcnt = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2905 cnt = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2906
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2907
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2908 /*
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2909 * check to see if we need to use the copy buffer for pages over
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2910 * the segment attr.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2911 */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2912 sglinfo->si_bounce_on_seg = B_FALSE;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2913 if (sglinfo->si_flags & _DDI_DMA_BOUNCE_ON_SEG) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2914 sglinfo->si_bounce_on_seg = rootnex_need_bounce_seg(
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2915 dmar_object, sglinfo);
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2916 }
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2917
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2918 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2919 * if we were passed down a linked list of pages, i.e. pointer to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2920 * page_t, use this to get our physical address and buf offset.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2921 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2922 if (buftype == DMA_OTYP_PAGES) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2923 pp = dmar_object->dmao_obj.pp_obj.pp_pp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2924 ASSERT(!PP_ISFREE(pp) && PAGE_LOCKED(pp));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2925 offset = dmar_object->dmao_obj.pp_obj.pp_offset &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2926 MMU_PAGEOFFSET;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	2927 paddr = pfn_to_pa(pp->p_pagenum) + offset;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2928 psize = MIN(size, (MMU_PAGESIZE - offset));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2929 pp = pp->p_next;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2930 sglinfo->si_asp = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2931
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2932 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2933 * We weren't passed down a linked list of pages, but if we were passed
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2934 * down an array of pages, use this to get our physical address and buf
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2935 * offset.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2936 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2937 } else if (pplist != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2938 ASSERT((buftype == DMA_OTYP_VADDR) \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2939 (buftype == DMA_OTYP_BUFVADDR));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2940
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2941 offset = (uintptr_t)vaddr & MMU_PAGEOFFSET;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2942 sglinfo->si_asp = dmar_object->dmao_obj.virt_obj.v_as;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2943 if (sglinfo->si_asp == NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2944 sglinfo->si_asp = &kas;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2945 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2946
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2947 ASSERT(!PP_ISFREE(pplist[pcnt]));
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	2948 paddr = pfn_to_pa(pplist[pcnt]->p_pagenum);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2949 paddr += offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2950 psize = MIN(size, (MMU_PAGESIZE - offset));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2951 pcnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2952
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2953 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2954 * All we have is a virtual address, we'll need to call into the VM
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2955 * to get the physical address.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2956 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2957 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2958 ASSERT((buftype == DMA_OTYP_VADDR) \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2959 (buftype == DMA_OTYP_BUFVADDR));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2960
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2961 offset = (uintptr_t)vaddr & MMU_PAGEOFFSET;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2962 sglinfo->si_asp = dmar_object->dmao_obj.virt_obj.v_as;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2963 if (sglinfo->si_asp == NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2964 sglinfo->si_asp = &kas;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2965 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2966
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	2967 paddr = pfn_to_pa(hat_getpfnum(sglinfo->si_asp->a_hat, vaddr));
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2968 paddr += offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2969 psize = MIN(size, (MMU_PAGESIZE - offset));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2970 vaddr += psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2971 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2972
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	2973 raddr = ROOTNEX_PADDR_TO_RBASE(paddr);
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	2974
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2975 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2976 * Setup the first cookie with the physical address of the page and the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2977 * size of the page (which takes into account the initial offset into
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2978 * the page.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2979 */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	2980 sgl[cnt].dmac_laddress = raddr;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2981 sgl[cnt].dmac_size = psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2982 sgl[cnt].dmac_type = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2983
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2984 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2985 * Save away the buffer offset into the page. We'll need this later in
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2986 * the copy buffer code to help figure out the page index within the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2987 * buffer and the offset into the current page.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2988 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2989 sglinfo->si_buf_offset = offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2990
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2991 /*
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2992 * If we are using the copy buffer for anything over the segment
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2993 * boundary, and this page is over the segment boundary.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2994 * OR
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2995 * if the DMA engine can't reach the physical address.
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	2996 */
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2997 if (((sglinfo->si_bounce_on_seg) &&
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2998 ((raddr + psize) > sglinfo->si_segmask)) \|\|
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	2999 ((raddr < addrlo) \|\| ((raddr + psize) > addrhi))) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3000 /*
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3001 * Increase how much copy buffer we use. We always increase by
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3002 * pagesize so we don't have to worry about converting offsets.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3003 * Set a flag in the cookies dmac_type to indicate that it uses
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3004 * the copy buffer. If this isn't the last cookie, go to the
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3005 * next cookie (since we separate each page which uses the copy
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3006 * buffer in case the copy buffer is not physically contiguous.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3007 */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3008 sglinfo->si_copybuf_req += MMU_PAGESIZE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3009 sgl[cnt].dmac_type = ROOTNEX_USES_COPYBUF;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3010 if ((cnt + 1) < sglinfo->si_max_pages) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3011 cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3012 sgl[cnt].dmac_laddress = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3013 sgl[cnt].dmac_size = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3014 sgl[cnt].dmac_type = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3015 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3016 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3017
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3018 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3019 * save this page's physical address so we can figure out if the next
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3020 * page is physically contiguous. Keep decrementing size until we are
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3021 * done with the buffer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3022 */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3023 last_page = raddr & MMU_PAGEMASK;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3024 size -= psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3025
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3026 while (size > 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3027 /* Get the size for this page (i.e. partial or full page) */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3028 psize = MIN(size, MMU_PAGESIZE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3029
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3030 if (buftype == DMA_OTYP_PAGES) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3031 /* get the paddr from the page_t */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3032 ASSERT(!PP_ISFREE(pp) && PAGE_LOCKED(pp));
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3033 paddr = pfn_to_pa(pp->p_pagenum);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3034 pp = pp->p_next;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3035 } else if (pplist != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3036 /* index into the array of page_t's to get the paddr */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3037 ASSERT(!PP_ISFREE(pplist[pcnt]));
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3038 paddr = pfn_to_pa(pplist[pcnt]->p_pagenum);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3039 pcnt++;
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	3040 } else {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3041 /* call into the VM to get the paddr */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3042 paddr = pfn_to_pa(hat_getpfnum(sglinfo->si_asp->a_hat,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3043 vaddr));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3044 vaddr += psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3045 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3046
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3047 raddr = ROOTNEX_PADDR_TO_RBASE(paddr);
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3048
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3049 /*
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3050 * If we are using the copy buffer for anything over the
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3051 * segment boundary, and this page is over the segment
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3052 * boundary.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3053 * OR
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3054 * if the DMA engine can't reach the physical address.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3055 */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3056 if (((sglinfo->si_bounce_on_seg) &&
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3057 ((raddr + psize) > sglinfo->si_segmask)) \|\|
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3058 ((raddr < addrlo) \|\| ((raddr + psize) > addrhi))) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3059
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3060 sglinfo->si_copybuf_req += MMU_PAGESIZE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3061
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	3062 /*
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3063 * if there is something in the current cookie, go to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3064 * the next one. We only want one page in a cookie which
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3065 * uses the copybuf since the copybuf doesn't have to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3066 * be physically contiguous.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3067 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3068 if (sgl[cnt].dmac_size != 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3069 cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3070 }
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3071 sgl[cnt].dmac_laddress = raddr;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3072 sgl[cnt].dmac_size = psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3073 #if defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3074 sgl[cnt].dmac_type = ROOTNEX_USES_COPYBUF;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3075 #else
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3076 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3077 * save the buf offset for 32-bit kernel. used in the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3078 * obsoleted interfaces.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3079 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3080 sgl[cnt].dmac_type = ROOTNEX_USES_COPYBUF \|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3081 (dmar_object->dmao_size - size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3082 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3083 /* if this isn't the last cookie, go to the next one */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3084 if ((cnt + 1) < sglinfo->si_max_pages) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3085 cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3086 sgl[cnt].dmac_laddress = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3087 sgl[cnt].dmac_size = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3088 sgl[cnt].dmac_type = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3089 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3090
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3091 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3092 * this page didn't need the copy buffer, if it's not physically
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3093 * contiguous, or it would put us over a segment boundary, or it
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3094 * puts us over the max cookie size, or the current sgl doesn't
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3095 * have anything in it.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3096 */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3097 } else if (((last_page + MMU_PAGESIZE) != raddr) \|\|
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3098 !(raddr & sglinfo->si_segmask) \|\|
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3099 ((sgl[cnt].dmac_size + psize) > maxseg) \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3100 (sgl[cnt].dmac_size == 0)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3101 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3102 * if we're not already in a new cookie, go to the next
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3103 * cookie.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3104 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3105 if (sgl[cnt].dmac_size != 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3106 cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3107 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3108
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3109 /* save the cookie information */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3110 sgl[cnt].dmac_laddress = raddr;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3111 sgl[cnt].dmac_size = psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3112 #if defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3113 sgl[cnt].dmac_type = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3114 #else
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3115 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3116 * save the buf offset for 32-bit kernel. used in the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3117 * obsoleted interfaces.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3118 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3119 sgl[cnt].dmac_type = dmar_object->dmao_size - size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3120 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3121
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3122 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3123 * this page didn't need the copy buffer, it is physically
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3124 * contiguous with the last page, and it's <= the max cookie
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3125 * size.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3126 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3127 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3128 sgl[cnt].dmac_size += psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3129
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3130 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3131 * if this exactly == the maximum cookie size, and
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3132 * it isn't the last cookie, go to the next cookie.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3133 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3134 if (((sgl[cnt].dmac_size + psize) == maxseg) &&
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3135 ((cnt + 1) < sglinfo->si_max_pages)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3136 cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3137 sgl[cnt].dmac_laddress = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3138 sgl[cnt].dmac_size = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3139 sgl[cnt].dmac_type = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3140 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3141 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3142
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3143 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3144 * save this page's physical address so we can figure out if the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3145 * next page is physically contiguous. Keep decrementing size
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3146 * until we are done with the buffer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3147 */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3148 last_page = raddr;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3149 size -= psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3150 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3151
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3152 /* we're done, save away how many cookies the sgl has */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3153 if (sgl[cnt].dmac_size == 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3154 ASSERT(cnt < sglinfo->si_max_pages);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3155 sglinfo->si_sgl_size = cnt;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3156 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3157 sglinfo->si_sgl_size = cnt + 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3158 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3159 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3160
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3161 static void
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3162 rootnex_dvma_get_sgl(ddi_dma_obj_t dmar_object, ddi_dma_cookie_t sgl,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3163 rootnex_sglinfo_t *sglinfo)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3164 {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3165 uint64_t offset;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3166 uint64_t maxseg;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3167 uint64_t dvaddr;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3168 struct dvmaseg *dvs;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3169 uint64_t paddr;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3170 uint32_t psize, ssize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3171 uint32_t size;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3172 uint_t cnt;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3173 int physcontig;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3174
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3175 ASSERT(dmar_object->dmao_type == DMA_OTYP_DVADDR);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3176
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3177 /* shortcuts */
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3178 maxseg = sglinfo->si_max_cookie_size;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3179 size = dmar_object->dmao_size;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3180
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3181 cnt = 0;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3182 sglinfo->si_bounce_on_seg = B_FALSE;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3183
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3184 dvs = dmar_object->dmao_obj.dvma_obj.dv_seg;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3185 offset = dmar_object->dmao_obj.dvma_obj.dv_off;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3186 ssize = dvs->dvs_len;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3187 paddr = dvs->dvs_start;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3188 paddr += offset;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3189 psize = MIN(ssize, (maxseg - offset));
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3190 dvaddr = paddr + psize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3191 ssize -= psize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3192
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3193 sgl[cnt].dmac_laddress = paddr;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3194 sgl[cnt].dmac_size = psize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3195 sgl[cnt].dmac_type = 0;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3196
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3197 size -= psize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3198 while (size > 0) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3199 if (ssize == 0) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3200 dvs++;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3201 ssize = dvs->dvs_len;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3202 dvaddr = dvs->dvs_start;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3203 physcontig = 0;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3204 } else
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3205 physcontig = 1;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3206
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3207 paddr = dvaddr;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3208 psize = MIN(ssize, maxseg);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3209 dvaddr += psize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3210 ssize -= psize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3211
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3212 if (!physcontig \|\| !(paddr & sglinfo->si_segmask) \|\|
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3213 ((sgl[cnt].dmac_size + psize) > maxseg) \|\|
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3214 (sgl[cnt].dmac_size == 0)) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3215 /*
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3216 * if we're not already in a new cookie, go to the next
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3217 * cookie.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3218 */
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3219 if (sgl[cnt].dmac_size != 0) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3220 cnt++;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3221 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3222
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3223 /* save the cookie information */
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3224 sgl[cnt].dmac_laddress = paddr;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3225 sgl[cnt].dmac_size = psize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3226 sgl[cnt].dmac_type = 0;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3227 } else {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3228 sgl[cnt].dmac_size += psize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3229
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3230 /*
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3231 * if this exactly == the maximum cookie size, and
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3232 * it isn't the last cookie, go to the next cookie.
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3233 */
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3234 if (((sgl[cnt].dmac_size + psize) == maxseg) &&
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3235 ((cnt + 1) < sglinfo->si_max_pages)) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3236 cnt++;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3237 sgl[cnt].dmac_laddress = 0;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3238 sgl[cnt].dmac_size = 0;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3239 sgl[cnt].dmac_type = 0;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3240 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3241 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3242 size -= psize;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3243 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3244
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3245 /* we're done, save away how many cookies the sgl has */
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3246 if (sgl[cnt].dmac_size == 0) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3247 sglinfo->si_sgl_size = cnt;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3248 } else {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3249 sglinfo->si_sgl_size = cnt + 1;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3250 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3251 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3252
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3253 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3254 * rootnex_bind_slowpath()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3255 * Call in the bind path if the calling driver can't use the sgl without
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3256 * modifying it. We either need to use the copy buffer and/or we will end up
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3257 * with a partial bind.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3258 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3259 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3260 rootnex_bind_slowpath(ddi_dma_impl_t hp, struct ddi_dma_req dmareq,
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3261 rootnex_dma_t dma, ddi_dma_attr_t attr, ddi_dma_obj_t *dmao, int kmflag)
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3262 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3263 rootnex_sglinfo_t *sinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3264 rootnex_window_t *window;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3265 ddi_dma_cookie_t *cookie;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3266 size_t copybuf_used;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3267 size_t dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3268 boolean_t partial;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3269 off_t cur_offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3270 page_t *cur_pp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3271 major_t mnum;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3272 int e;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3273 int i;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3274
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3275
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3276 sinfo = &dma->dp_sglinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3277 copybuf_used = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3278 partial = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3279
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3280 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3281 * If we're using the copybuf, set the copybuf state in dma struct.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3282 * Needs to be first since it sets the copy buffer size.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3283 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3284 if (sinfo->si_copybuf_req != 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3285 e = rootnex_setup_copybuf(hp, dmareq, dma, attr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3286 if (e != DDI_SUCCESS) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3287 return (e);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3288 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3289 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3290 dma->dp_copybuf_size = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3291 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3292
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3293 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3294 * Figure out if we need to do a partial mapping. If so, figure out
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3295 * if we need to trim the buffers when we munge the sgl.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3296 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3297 if ((dma->dp_copybuf_size < sinfo->si_copybuf_req) \|\|
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3298 (dmao->dmao_size > dma->dp_maxxfer) \|\|
13364 b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	3299 ((unsigned)attr->dma_attr_sgllen < sinfo->si_sgl_size)) {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3300 dma->dp_partial_required = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3301 if (attr->dma_attr_granular != 1) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3302 dma->dp_trim_required = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3303 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3304 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3305 dma->dp_partial_required = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3306 dma->dp_trim_required = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3307 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3308
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3309 /* If we need to do a partial bind, make sure the driver supports it */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3310 if (dma->dp_partial_required &&
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3311 !(dmareq->dmar_flags & DDI_DMA_PARTIAL)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3312
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3313 mnum = ddi_driver_major(dma->dp_dip);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3314 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3315 * patchable which allows us to print one warning per major
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3316 * number.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3317 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3318 if ((rootnex_bind_warn) &&
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3319 ((rootnex_warn_list[mnum] & ROOTNEX_BIND_WARNING) == 0)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3320 rootnex_warn_list[mnum] \|= ROOTNEX_BIND_WARNING;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3321 cmn_err(CE_WARN, "!%s: coding error detected, the "
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3322 "driver is using ddi_dma_attr(9S) incorrectly. "
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3323 "There is a small risk of data corruption in "
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3324 "particular with large I/Os. The driver should be "
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3325 "replaced with a corrected version for proper "
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3326 "system operation. To disable this warning, add "
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3327 "'set rootnex:rootnex_bind_warn=0' to "
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3328 "/etc/system(4).", ddi_driver_name(dma->dp_dip));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3329 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3330 return (DDI_DMA_TOOBIG);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3331 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3332
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3333 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3334 * we might need multiple windows, setup state to handle them. In this
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3335 * code path, we will have at least one window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3336 */
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3337 e = rootnex_setup_windows(hp, dma, attr, dmao, kmflag);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3338 if (e != DDI_SUCCESS) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3339 rootnex_teardown_copybuf(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3340 return (e);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3341 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3342
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3343 window = &dma->dp_window[0];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3344 cookie = &dma->dp_cookies[0];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3345 cur_offset = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3346 rootnex_init_win(hp, dma, window, cookie, cur_offset);
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3347 if (dmao->dmao_type == DMA_OTYP_PAGES) {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3348 cur_pp = dmareq->dmar_object.dmao_obj.pp_obj.pp_pp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3349 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3350
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3351 /* loop though all the cookies we got back from get_sgl() */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3352 for (i = 0; i < sinfo->si_sgl_size; i++) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3353 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3354 * If we're using the copy buffer, check this cookie and setup
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3355 * its associated copy buffer state. If this cookie uses the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3356 * copy buffer, make sure we sync this window during dma_sync.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3357 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3358 if (dma->dp_copybuf_size > 0) {
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3359 rootnex_setup_cookie(dmao, dma, cookie,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3360 cur_offset, &copybuf_used, &cur_pp);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3361 if (cookie->dmac_type & ROOTNEX_USES_COPYBUF) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3362 window->wd_dosync = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3363 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3364 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3365
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3366 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3367 * save away the cookie size, since it could be modified in
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3368 * the windowing code.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3369 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3370 dmac_size = cookie->dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3371
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3372 /* if we went over max copybuf size */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3373 if (dma->dp_copybuf_size &&
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3374 (copybuf_used > dma->dp_copybuf_size)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3375 partial = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3376 e = rootnex_copybuf_window_boundary(hp, dma, &window,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3377 cookie, cur_offset, &copybuf_used);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3378 if (e != DDI_SUCCESS) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3379 rootnex_teardown_copybuf(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3380 rootnex_teardown_windows(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3381 return (e);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3382 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3383
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3384 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3385 * if the coookie uses the copy buffer, make sure the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3386 * new window we just moved to is set to sync.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3387 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3388 if (cookie->dmac_type & ROOTNEX_USES_COPYBUF) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3389 window->wd_dosync = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3390 }
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	3391 ROOTNEX_DPROBE1(rootnex__copybuf__window, dev_info_t *,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3392 dma->dp_dip);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3393
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3394 /* if the cookie cnt == max sgllen, move to the next window */
13364 b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	3395 } else if (window->wd_cookie_cnt >=
b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	3396 (unsigned)attr->dma_attr_sgllen) {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3397 partial = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3398 ASSERT(window->wd_cookie_cnt == attr->dma_attr_sgllen);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3399 e = rootnex_sgllen_window_boundary(hp, dma, &window,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3400 cookie, attr, cur_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3401 if (e != DDI_SUCCESS) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3402 rootnex_teardown_copybuf(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3403 rootnex_teardown_windows(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3404 return (e);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3405 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3406
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3407 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3408 * if the coookie uses the copy buffer, make sure the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3409 * new window we just moved to is set to sync.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3410 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3411 if (cookie->dmac_type & ROOTNEX_USES_COPYBUF) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3412 window->wd_dosync = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3413 }
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	3414 ROOTNEX_DPROBE1(rootnex__sgllen__window, dev_info_t *,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3415 dma->dp_dip);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3416
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3417 /* else if we will be over maxxfer */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3418 } else if ((window->wd_size + dmac_size) >
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3419 dma->dp_maxxfer) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3420 partial = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3421 e = rootnex_maxxfer_window_boundary(hp, dma, &window,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3422 cookie);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3423 if (e != DDI_SUCCESS) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3424 rootnex_teardown_copybuf(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3425 rootnex_teardown_windows(dma);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3426 return (e);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3427 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3428
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3429 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3430 * if the coookie uses the copy buffer, make sure the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3431 * new window we just moved to is set to sync.
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	3432 */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3433 if (cookie->dmac_type & ROOTNEX_USES_COPYBUF) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3434 window->wd_dosync = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3435 }
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	3436 ROOTNEX_DPROBE1(rootnex__maxxfer__window, dev_info_t *,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3437 dma->dp_dip);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3438
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3439 /* else this cookie fits in the current window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3440 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3441 window->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3442 window->wd_size += dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3443 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3444
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3445 /* track our offset into the buffer, go to the next cookie */
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3446 ASSERT(dmac_size <= dmao->dmao_size);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3447 ASSERT(cookie->dmac_size <= dmac_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3448 cur_offset += dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3449 cookie++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3450 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3451
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3452 /* if we ended up with a zero sized window in the end, clean it up */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3453 if (window->wd_size == 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3454 hp->dmai_nwin--;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3455 window--;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3456 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3457
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3458 ASSERT(window->wd_trim.tr_trim_last == B_FALSE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3459
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3460 if (!partial) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3461 return (DDI_DMA_MAPPED);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3462 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3463
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3464 ASSERT(dma->dp_partial_required);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3465 return (DDI_DMA_PARTIAL_MAP);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3466 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3467
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3468 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3469 * rootnex_setup_copybuf()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3470 * Called in bind slowpath. Figures out if we're going to use the copy
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3471 * buffer, and if we do, sets up the basic state to handle it.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3472 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3473 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3474 rootnex_setup_copybuf(ddi_dma_impl_t hp, struct ddi_dma_req dmareq,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3475 rootnex_dma_t dma, ddi_dma_attr_t attr)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3476 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3477 rootnex_sglinfo_t *sinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3478 ddi_dma_attr_t lattr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3479 size_t max_copybuf;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3480 int cansleep;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3481 int e;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3482 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3483 int vmflag;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3484 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3485
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3486 ASSERT(!dma->dp_dvma_used);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3487
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3488 sinfo = &dma->dp_sglinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3489
5251 aad4f9e16063 PSARC/2007/589 dma-max-arch scsi capability mrj parents: 5084 diff changeset	3490 /* read this first so it's consistent through the routine */
aad4f9e16063 PSARC/2007/589 dma-max-arch scsi capability mrj parents: 5084 diff changeset	3491 max_copybuf = i_ddi_copybuf_size() & MMU_PAGEMASK;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3492
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3493 /* We need to call into the rootnex on ddi_dma_sync() */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3494 hp->dmai_rflags &= ~DMP_NOSYNC;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3495
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3496 /* make sure the copybuf size <= the max size */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3497 dma->dp_copybuf_size = MIN(sinfo->si_copybuf_req, max_copybuf);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3498 ASSERT((dma->dp_copybuf_size & MMU_PAGEOFFSET) == 0);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3499
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3500 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3501 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3502 * if we don't have kva space to copy to/from, allocate the KVA space
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3503 * now. We only do this for the 32-bit kernel. We use seg kpm space for
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3504 * the 64-bit kernel.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3505 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3506 if ((dmareq->dmar_object.dmao_type == DMA_OTYP_PAGES) \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3507 (dmareq->dmar_object.dmao_obj.virt_obj.v_as != NULL)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3508
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3509 /* convert the sleep flags */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3510 if (dmareq->dmar_fp == DDI_DMA_SLEEP) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3511 vmflag = VM_SLEEP;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3512 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3513 vmflag = VM_NOSLEEP;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3514 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3515
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3516 /* allocate Kernel VA space that we can bcopy to/from */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3517 dma->dp_kva = vmem_alloc(heap_arena, dma->dp_copybuf_size,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3518 vmflag);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3519 if (dma->dp_kva == NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3520 return (DDI_DMA_NORESOURCES);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3521 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3522 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3523 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3524
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3525 /* convert the sleep flags */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3526 if (dmareq->dmar_fp == DDI_DMA_SLEEP) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3527 cansleep = 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3528 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3529 cansleep = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3530 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3531
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3532 /*
7173 fbb50387fa5c 6728330 ide driver can hang in dom0 mrj parents: 5591 diff changeset	3533 * Allocate the actual copy buffer. This needs to fit within the DMA
fbb50387fa5c 6728330 ide driver can hang in dom0 mrj parents: 5591 diff changeset	3534 * engine limits, so we can't use kmem_alloc... We don't need
fbb50387fa5c 6728330 ide driver can hang in dom0 mrj parents: 5591 diff changeset	3535 * contiguous memory (sgllen) since we will be forcing windows on
fbb50387fa5c 6728330 ide driver can hang in dom0 mrj parents: 5591 diff changeset	3536 * sgllen anyway.
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3537 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3538 lattr = *attr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3539 lattr.dma_attr_align = MMU_PAGESIZE;
13364 b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	3540 lattr.dma_attr_sgllen = -1; /* no limit */
11793 9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3541 /*
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3542 * if we're using the copy buffer because of seg, use that for our
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3543 * upper address limit.
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3544 */
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3545 if (sinfo->si_bounce_on_seg) {
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3546 lattr.dma_attr_addr_hi = lattr.dma_attr_seg;
9e2782423ae1 6908360 49.3% snv_129 vdb407_nvSeqWriteBs128kFs1g_zfs-raidz performance regression x86 Mark Johnson <Mark.Johnson@Sun.COM> parents: 11600 diff changeset	3547 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3548 e = i_ddi_mem_alloc(dma->dp_dip, &lattr, dma->dp_copybuf_size, cansleep,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3549 0, NULL, &dma->dp_cbaddr, &dma->dp_cbsize, NULL);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3550 if (e != DDI_SUCCESS) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3551 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3552 if (dma->dp_kva != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3553 vmem_free(heap_arena, dma->dp_kva,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3554 dma->dp_copybuf_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3555 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3556 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3557 return (DDI_DMA_NORESOURCES);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3558 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3559
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	3560 ROOTNEX_DPROBE2(rootnex__alloc__copybuf, dev_info_t *, dma->dp_dip,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3561 size_t, dma->dp_copybuf_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3562
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3563 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3564 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3565
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3566
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3567 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3568 * rootnex_setup_windows()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3569 * Called in bind slowpath to setup the window state. We always have windows
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3570 * in the slowpath. Even if the window count = 1.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3571 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3572 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3573 rootnex_setup_windows(ddi_dma_impl_t hp, rootnex_dma_t dma,
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3574 ddi_dma_attr_t attr, ddi_dma_obj_t dmao, int kmflag)
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3575 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3576 rootnex_window_t *windowp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3577 rootnex_sglinfo_t *sinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3578 size_t copy_state_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3579 size_t win_state_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3580 size_t state_available;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3581 size_t space_needed;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3582 uint_t copybuf_win;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3583 uint_t maxxfer_win;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3584 size_t space_used;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3585 uint_t sglwin;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3586
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3587
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3588 sinfo = &dma->dp_sglinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3589
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3590 dma->dp_current_win = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3591 hp->dmai_nwin = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3592
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3593 /* If we don't need to do a partial, we only have one window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3594 if (!dma->dp_partial_required) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3595 dma->dp_max_win = 1;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3596
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3597 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3598 * we need multiple windows, need to figure out the worse case number
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3599 * of windows.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3600 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3601 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3602 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3603 * if we need windows because we need more copy buffer that
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3604 * we allow, the worse case number of windows we could need
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3605 * here would be (copybuf space required / copybuf space that
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3606 * we have) plus one for remainder, and plus 2 to handle the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3607 * extra pages on the trim for the first and last pages of the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3608 * buffer (a page is the minimum window size so under the right
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3609 * attr settings, you could have a window for each page).
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3610 * The last page will only be hit here if the size is not a
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3611 * multiple of the granularity (which theoretically shouldn't
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3612 * be the case but never has been enforced, so we could have
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3613 * broken things without it).
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3614 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3615 if (sinfo->si_copybuf_req > dma->dp_copybuf_size) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3616 ASSERT(dma->dp_copybuf_size > 0);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3617 copybuf_win = (sinfo->si_copybuf_req /
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3618 dma->dp_copybuf_size) + 1 + 2;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3619 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3620 copybuf_win = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3621 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3622
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3623 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3624 * if we need windows because we have more cookies than the H/W
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3625 * can handle, the number of windows we would need here would
12118 6928a154b915 6924464 fstyp on kimchee device panic the system Mark Johnson <mark.r.johnson@oracle.com> parents: 12027 diff changeset	3626 * be (cookie count / cookies count H/W supports minus 1[for
6928a154b915 6924464 fstyp on kimchee device panic the system Mark Johnson <mark.r.johnson@oracle.com> parents: 12027 diff changeset	3627 * trim]) plus one for remainder.
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3628 */
13364 b600b8d927a8 696 Incorrect check dma_attr_sgllen <= 0 in rootnex_valid_alloc_parms Garrett D'Amore <garrett@nexenta.com> parents: 13268 diff changeset	3629 if ((unsigned)attr->dma_attr_sgllen < sinfo->si_sgl_size) {
12118 6928a154b915 6924464 fstyp on kimchee device panic the system Mark Johnson <mark.r.johnson@oracle.com> parents: 12027 diff changeset	3630 sglwin = (sinfo->si_sgl_size /
6928a154b915 6924464 fstyp on kimchee device panic the system Mark Johnson <mark.r.johnson@oracle.com> parents: 12027 diff changeset	3631 (attr->dma_attr_sgllen - 1)) + 1;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3632 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3633 sglwin = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3634 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3635
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3636 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3637 * if we need windows because we're binding more memory than the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3638 * H/W can transfer at once, the number of windows we would need
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3639 * here would be (xfer count / max xfer H/W supports) plus one
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3640 * for remainder, and plus 2 to handle the extra pages on the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3641 * trim (see above comment about trim)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3642 */
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3643 if (dmao->dmao_size > dma->dp_maxxfer) {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3644 maxxfer_win = (dmao->dmao_size /
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3645 dma->dp_maxxfer) + 1 + 2;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3646 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3647 maxxfer_win = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3648 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3649 dma->dp_max_win = copybuf_win + sglwin + maxxfer_win;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3650 ASSERT(dma->dp_max_win > 0);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3651 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3652 win_state_size = dma->dp_max_win * sizeof (rootnex_window_t);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3653
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3654 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3655 * Get space for window and potential copy buffer state. Before we
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3656 * go and allocate memory, see if we can get away with using what's
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3657 * left in the pre-allocted state or the dynamically allocated sgl.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3658 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3659 space_used = (uintptr_t)(sinfo->si_sgl_size *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3660 sizeof (ddi_dma_cookie_t));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3661
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3662 /* if we dynamically allocated space for the cookies */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3663 if (dma->dp_need_to_free_cookie) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3664 /* if we have more space in the pre-allocted buffer, use it */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3665 ASSERT(space_used <= dma->dp_cookie_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3666 if ((dma->dp_cookie_size - space_used) <=
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3667 rootnex_state->r_prealloc_size) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3668 state_available = rootnex_state->r_prealloc_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3669 windowp = (rootnex_window_t *)dma->dp_prealloc_buffer;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3670
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3671 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3672 * else, we have more free space in the dynamically allocated
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3673 * buffer, i.e. the buffer wasn't worse case fragmented so we
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3674 * didn't need a lot of cookies.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3675 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3676 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3677 state_available = dma->dp_cookie_size - space_used;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3678 windowp = (rootnex_window_t *)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3679 &dma->dp_cookies[sinfo->si_sgl_size];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3680 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3681
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3682 /* we used the pre-alloced buffer */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3683 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3684 ASSERT(space_used <= rootnex_state->r_prealloc_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3685 state_available = rootnex_state->r_prealloc_size - space_used;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3686 windowp = (rootnex_window_t *)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3687 &dma->dp_cookies[sinfo->si_sgl_size];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3688 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3689
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3690 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3691 * figure out how much state we need to track the copy buffer. Add an
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3692 * addition 8 bytes for pointer alignemnt later.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3693 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3694 if (dma->dp_copybuf_size > 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3695 copy_state_size = sinfo->si_max_pages *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3696 sizeof (rootnex_pgmap_t);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3697 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3698 copy_state_size = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3699 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3700 /* add an additional 8 bytes for pointer alignment */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3701 space_needed = win_state_size + copy_state_size + 0x8;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3702
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3703 /* if we have enough space already, use it */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3704 if (state_available >= space_needed) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3705 dma->dp_window = windowp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3706 dma->dp_need_to_free_window = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3707
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3708 /* not enough space, need to allocate more. */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3709 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3710 dma->dp_window = kmem_alloc(space_needed, kmflag);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3711 if (dma->dp_window == NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3712 return (DDI_DMA_NORESOURCES);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3713 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3714 dma->dp_need_to_free_window = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3715 dma->dp_window_size = space_needed;
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	3716 ROOTNEX_DPROBE2(rootnex__bind__sp__alloc, dev_info_t *,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3717 dma->dp_dip, size_t, space_needed);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3718 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3719
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3720 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3721 * we allocate copy buffer state and window state at the same time.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3722 * setup our copy buffer state pointers. Make sure it's aligned.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3723 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3724 if (dma->dp_copybuf_size > 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3725 dma->dp_pgmap = (rootnex_pgmap_t *)(((uintptr_t)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3726 &dma->dp_window[dma->dp_max_win] + 0x7) & ~0x7);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3727
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3728 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3729 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3730 * make sure all pm_mapped, pm_vaddr, and pm_pp are set to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3731 * false/NULL. Should be quicker to bzero vs loop and set.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3732 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3733 bzero(dma->dp_pgmap, copy_state_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3734 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3735 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3736 dma->dp_pgmap = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3737 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3738
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3739 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3740 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3741
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3742
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3743 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3744 * rootnex_teardown_copybuf()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3745 * cleans up after rootnex_setup_copybuf()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3746 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3747 static void
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3748 rootnex_teardown_copybuf(rootnex_dma_t *dma)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3749 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3750 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3751 int i;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3752
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3753 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3754 * if we allocated kernel heap VMEM space, go through all the pages and
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3755 * map out any of the ones that we're mapped into the kernel heap VMEM
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3756 * arena. Then free the VMEM space.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3757 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3758 if (dma->dp_kva != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3759 for (i = 0; i < dma->dp_sglinfo.si_max_pages; i++) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3760 if (dma->dp_pgmap[i].pm_mapped) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3761 hat_unload(kas.a_hat, dma->dp_pgmap[i].pm_kaddr,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3762 MMU_PAGESIZE, HAT_UNLOAD);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3763 dma->dp_pgmap[i].pm_mapped = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3764 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3765 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3766
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3767 vmem_free(heap_arena, dma->dp_kva, dma->dp_copybuf_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3768 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3769
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3770 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3771
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3772 /* if we allocated a copy buffer, free it */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3773 if (dma->dp_cbaddr != NULL) {
1900 7657500f392a PSARC 2006/188 Memory Cache Disable Support on x86 eota parents: 1865 diff changeset	3774 i_ddi_mem_free(dma->dp_cbaddr, NULL);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3775 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3776 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3777
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3778
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3779 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3780 * rootnex_teardown_windows()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3781 * cleans up after rootnex_setup_windows()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3782 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3783 static void
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3784 rootnex_teardown_windows(rootnex_dma_t *dma)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3785 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3786 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3787 * if we had to allocate window state on the last bind (because we
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3788 * didn't have enough pre-allocated space in the handle), free it.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3789 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3790 if (dma->dp_need_to_free_window) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3791 kmem_free(dma->dp_window, dma->dp_window_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3792 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3793 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3794
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3795
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3796 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3797 * rootnex_init_win()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3798 * Called in bind slow path during creation of a new window. Initializes
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3799 * window state to default values.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3800 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3801 /ARGSUSED/
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3802 static void
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3803 rootnex_init_win(ddi_dma_impl_t hp, rootnex_dma_t dma,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3804 rootnex_window_t window, ddi_dma_cookie_t cookie, off_t cur_offset)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3805 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3806 hp->dmai_nwin++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3807 window->wd_dosync = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3808 window->wd_offset = cur_offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3809 window->wd_size = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3810 window->wd_first_cookie = cookie;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3811 window->wd_cookie_cnt = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3812 window->wd_trim.tr_trim_first = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3813 window->wd_trim.tr_trim_last = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3814 window->wd_trim.tr_first_copybuf_win = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3815 window->wd_trim.tr_last_copybuf_win = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3816 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3817 window->wd_remap_copybuf = dma->dp_cb_remaping;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3818 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3819 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3820
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3821
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3822 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3823 * rootnex_setup_cookie()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3824 * Called in the bind slow path when the sgl uses the copy buffer. If any of
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3825 * the sgl uses the copy buffer, we need to go through each cookie, figure
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3826 * out if it uses the copy buffer, and if it does, save away everything we'll
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3827 * need during sync.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3828 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3829 static void
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3830 rootnex_setup_cookie(ddi_dma_obj_t dmar_object, rootnex_dma_t dma,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3831 ddi_dma_cookie_t cookie, off_t cur_offset, size_t copybuf_used,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3832 page_t **cur_pp)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3833 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3834 boolean_t copybuf_sz_power_2;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3835 rootnex_sglinfo_t *sinfo;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3836 paddr_t paddr;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3837 uint_t pidx;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3838 uint_t pcnt;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3839 off_t poff;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3840 #if defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3841 pfn_t pfn;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3842 #else
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3843 page_t **pplist;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3844 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3845
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3846 ASSERT(dmar_object->dmao_type != DMA_OTYP_DVADDR);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3847
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3848 sinfo = &dma->dp_sglinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3849
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3850 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3851 * Calculate the page index relative to the start of the buffer. The
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3852 * index to the current page for our buffer is the offset into the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3853 * first page of the buffer plus our current offset into the buffer
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3854 * itself, shifted of course...
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3855 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3856 pidx = (sinfo->si_buf_offset + cur_offset) >> MMU_PAGESHIFT;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3857 ASSERT(pidx < sinfo->si_max_pages);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3858
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3859 /* if this cookie uses the copy buffer */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3860 if (cookie->dmac_type & ROOTNEX_USES_COPYBUF) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3861 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3862 * NOTE: we know that since this cookie uses the copy buffer, it
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3863 * is <= MMU_PAGESIZE.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3864 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3865
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3866 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3867 * get the offset into the page. For the 64-bit kernel, get the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3868 * pfn which we'll use with seg kpm.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3869 */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3870 poff = cookie->dmac_laddress & MMU_PAGEOFFSET;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3871 #if defined(__amd64)
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3872 /* mfn_to_pfn() is a NOP on i86pc */
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3873 pfn = mfn_to_pfn(cookie->dmac_laddress >> MMU_PAGESHIFT);
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3874 #endif /* __amd64 */
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3875
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3876 /* figure out if the copybuf size is a power of 2 */
14707 bcf4bfbd1b70 5255 uts shouldn't open-code ISP2 Josef 'Jeff' Sipek <josef.sipek@nexenta.com> parents: 13636 diff changeset	3877 if (!ISP2(dma->dp_copybuf_size)) {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3878 copybuf_sz_power_2 = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3879 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3880 copybuf_sz_power_2 = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3881 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3882
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3883 /* This page uses the copy buffer */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3884 dma->dp_pgmap[pidx].pm_uses_copybuf = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3885
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3886 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3887 * save the copy buffer KVA that we'll use with this page.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3888 * if we still fit within the copybuf, it's a simple add.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3889 * otherwise, we need to wrap over using & or % accordingly.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3890 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3891 if ((*copybuf_used + MMU_PAGESIZE) <= dma->dp_copybuf_size) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3892 dma->dp_pgmap[pidx].pm_cbaddr = dma->dp_cbaddr +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3893 *copybuf_used;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3894 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3895 if (copybuf_sz_power_2) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3896 dma->dp_pgmap[pidx].pm_cbaddr = (caddr_t)(
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3897 (uintptr_t)dma->dp_cbaddr +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3898 (*copybuf_used &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3899 (dma->dp_copybuf_size - 1)));
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	3900 } else {
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3901 dma->dp_pgmap[pidx].pm_cbaddr = (caddr_t)(
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3902 (uintptr_t)dma->dp_cbaddr +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3903 (*copybuf_used % dma->dp_copybuf_size));
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	3904 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3905 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3906
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3907 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3908 * over write the cookie physical address with the address of
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3909 * the physical address of the copy buffer page that we will
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3910 * use.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3911 */
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3912 paddr = pfn_to_pa(hat_getpfnum(kas.a_hat,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3913 dma->dp_pgmap[pidx].pm_cbaddr)) + poff;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3914
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	3915 cookie->dmac_laddress = ROOTNEX_PADDR_TO_RBASE(paddr);
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	3916
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3917 /* if we have a kernel VA, it's easy, just save that address */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3918 if ((dmar_object->dmao_type != DMA_OTYP_PAGES) &&
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3919 (sinfo->si_asp == &kas)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3920 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3921 * save away the page aligned virtual address of the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3922 * driver buffer. Offsets are handled in the sync code.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3923 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3924 dma->dp_pgmap[pidx].pm_kaddr = (caddr_t)(((uintptr_t)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3925 dmar_object->dmao_obj.virt_obj.v_addr + cur_offset)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3926 & MMU_PAGEMASK);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3927 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3928 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3929 * we didn't need to, and will never need to map this
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3930 * page.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3931 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3932 dma->dp_pgmap[pidx].pm_mapped = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3933 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3934
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3935 /* we don't have a kernel VA. We need one for the bcopy. */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3936 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3937 #if defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3938 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3939 * for the 64-bit kernel, it's easy. We use seg kpm to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3940 * get a Kernel VA for the corresponding pfn.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3941 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3942 dma->dp_pgmap[pidx].pm_kaddr = hat_kpm_pfn2va(pfn);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3943 #else
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3944 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3945 * for the 32-bit kernel, this is a pain. First we'll
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3946 * save away the page_t or user VA for this page. This
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3947 * is needed in rootnex_dma_win() when we switch to a
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3948 * new window which requires us to re-map the copy
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3949 * buffer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3950 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3951 pplist = dmar_object->dmao_obj.virt_obj.v_priv;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3952 if (dmar_object->dmao_type == DMA_OTYP_PAGES) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3953 dma->dp_pgmap[pidx].pm_pp = *cur_pp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3954 dma->dp_pgmap[pidx].pm_vaddr = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3955 } else if (pplist != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3956 dma->dp_pgmap[pidx].pm_pp = pplist[pidx];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3957 dma->dp_pgmap[pidx].pm_vaddr = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3958 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3959 dma->dp_pgmap[pidx].pm_pp = NULL;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3960 dma->dp_pgmap[pidx].pm_vaddr = (caddr_t)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3961 (((uintptr_t)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3962 dmar_object->dmao_obj.virt_obj.v_addr +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3963 cur_offset) & MMU_PAGEMASK);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3964 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3965
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3966 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3967 * save away the page aligned virtual address which was
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3968 * allocated from the kernel heap arena (taking into
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3969 * account if we need more copy buffer than we alloced
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3970 * and use multiple windows to handle this, i.e. &,%).
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3971 * NOTE: there isn't and physical memory backing up this
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3972 * virtual address space currently.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3973 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3974 if ((*copybuf_used + MMU_PAGESIZE) <=
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3975 dma->dp_copybuf_size) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3976 dma->dp_pgmap[pidx].pm_kaddr = (caddr_t)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3977 (((uintptr_t)dma->dp_kva + *copybuf_used) &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3978 MMU_PAGEMASK);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3979 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3980 if (copybuf_sz_power_2) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3981 dma->dp_pgmap[pidx].pm_kaddr = (caddr_t)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3982 (((uintptr_t)dma->dp_kva +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3983 (*copybuf_used &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3984 (dma->dp_copybuf_size - 1))) &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3985 MMU_PAGEMASK);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3986 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3987 dma->dp_pgmap[pidx].pm_kaddr = (caddr_t)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3988 (((uintptr_t)dma->dp_kva +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3989 (*copybuf_used %
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3990 dma->dp_copybuf_size)) &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3991 MMU_PAGEMASK);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3992 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3993 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3994
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3995 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3996 * if we haven't used up the available copy buffer yet,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3997 * map the kva to the physical page.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3998 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	3999 if (!dma->dp_cb_remaping && ((*copybuf_used +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4000 MMU_PAGESIZE) <= dma->dp_copybuf_size)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4001 dma->dp_pgmap[pidx].pm_mapped = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4002 if (dma->dp_pgmap[pidx].pm_pp != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4003 i86_pp_map(dma->dp_pgmap[pidx].pm_pp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4004 dma->dp_pgmap[pidx].pm_kaddr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4005 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4006 i86_va_map(dma->dp_pgmap[pidx].pm_vaddr,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4007 sinfo->si_asp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4008 dma->dp_pgmap[pidx].pm_kaddr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4009 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4010
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4011 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4012 * we've used up the available copy buffer, this page
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4013 * will have to be mapped during rootnex_dma_win() when
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4014 * we switch to a new window which requires a re-map
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4015 * the copy buffer. (32-bit kernel only)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4016 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4017 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4018 dma->dp_pgmap[pidx].pm_mapped = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4019 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4020 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4021 /* go to the next page_t */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4022 if (dmar_object->dmao_type == DMA_OTYP_PAGES) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4023 cur_pp = (cur_pp)->p_next;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4024 }
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	4025 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4026
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4027 /* add to the copy buffer count */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4028 *copybuf_used += MMU_PAGESIZE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4029
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4030 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4031 * This cookie doesn't use the copy buffer. Walk through the pages this
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4032 * cookie occupies to reflect this.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4033 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4034 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4035 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4036 * figure out how many pages the cookie occupies. We need to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4037 * use the original page offset of the buffer and the cookies
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4038 * offset in the buffer to do this.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4039 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4040 poff = (sinfo->si_buf_offset + cur_offset) & MMU_PAGEOFFSET;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4041 pcnt = mmu_btopr(cookie->dmac_size + poff);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4042
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4043 while (pcnt > 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4044 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4045 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4046 * the 32-bit kernel doesn't have seg kpm, so we need
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4047 * to map in the driver buffer (if it didn't come down
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4048 * with a kernel VA) on the fly. Since this page doesn't
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4049 * use the copy buffer, it's not, or will it ever, have
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4050 * to be mapped in.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4051 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4052 dma->dp_pgmap[pidx].pm_mapped = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4053 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4054 dma->dp_pgmap[pidx].pm_uses_copybuf = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4055
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4056 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4057 * we need to update pidx and cur_pp or we'll loose
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4058 * track of where we are.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4059 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4060 if (dmar_object->dmao_type == DMA_OTYP_PAGES) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4061 cur_pp = (cur_pp)->p_next;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4062 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4063 pidx++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4064 pcnt--;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4065 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4066 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4067 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4068
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4069
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4070 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4071 * rootnex_sgllen_window_boundary()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4072 * Called in the bind slow path when the next cookie causes us to exceed (in
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4073 * this case == since we start at 0 and sgllen starts at 1) the maximum sgl
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4074 * length supported by the DMA H/W.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4075 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4076 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4077 rootnex_sgllen_window_boundary(ddi_dma_impl_t hp, rootnex_dma_t dma,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4078 rootnex_window_t *windowp, ddi_dma_cookie_t cookie, ddi_dma_attr_t *attr,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4079 off_t cur_offset)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4080 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4081 off_t new_offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4082 size_t trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4083 off_t coffset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4084
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4085
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4086 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4087 * if we know we'll never have to trim, it's pretty easy. Just move to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4088 * the next window and init it. We're done.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4089 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4090 if (!dma->dp_trim_required) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4091 (*windowp)++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4092 rootnex_init_win(hp, dma, *windowp, cookie, cur_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4093 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4094 (*windowp)->wd_size = cookie->dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4095 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4096 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4097
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4098 /* figure out how much we need to trim from the window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4099 ASSERT(attr->dma_attr_granular != 0);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4100 if (dma->dp_granularity_power_2) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4101 trim_sz = (*windowp)->wd_size & (attr->dma_attr_granular - 1);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4102 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4103 trim_sz = (*windowp)->wd_size % attr->dma_attr_granular;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4104 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4105
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4106 /* The window's a whole multiple of granularity. We're done */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4107 if (trim_sz == 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4108 (*windowp)++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4109 rootnex_init_win(hp, dma, *windowp, cookie, cur_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4110 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4111 (*windowp)->wd_size = cookie->dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4112 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4113 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4114
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4115 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4116 * The window's not a whole multiple of granularity, since we know this
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4117 * is due to the sgllen, we need to go back to the last cookie and trim
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4118 * that one, add the left over part of the old cookie into the new
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4119 * window, and then add in the new cookie into the new window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4120 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4121
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4122 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4123 * make sure the driver isn't making us do something bad... Trimming and
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4124 * sgllen == 1 don't go together.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4125 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4126 if (attr->dma_attr_sgllen == 1) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4127 return (DDI_DMA_NOMAPPING);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4128 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4129
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4130 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4131 * first, setup the current window to account for the trim. Need to go
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4132 * back to the last cookie for this.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4133 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4134 cookie--;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4135 (*windowp)->wd_trim.tr_trim_last = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4136 (*windowp)->wd_trim.tr_last_cookie = cookie;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4137 (*windowp)->wd_trim.tr_last_paddr = cookie->dmac_laddress;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4138 ASSERT(cookie->dmac_size > trim_sz);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4139 (*windowp)->wd_trim.tr_last_size = cookie->dmac_size - trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4140 (*windowp)->wd_size -= trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4141
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4142 /* save the buffer offsets for the next window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4143 coffset = cookie->dmac_size - trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4144 new_offset = (windowp)->wd_offset + (windowp)->wd_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4145
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4146 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4147 * set this now in case this is the first window. all other cases are
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4148 * set in dma_win()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4149 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4150 cookie->dmac_size = (*windowp)->wd_trim.tr_last_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4151
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4152 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4153 * initialize the next window using what's left over in the previous
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4154 * cookie.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4155 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4156 (*windowp)++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4157 rootnex_init_win(hp, dma, *windowp, cookie, new_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4158 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4159 (*windowp)->wd_trim.tr_trim_first = B_TRUE;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4160 (*windowp)->wd_trim.tr_first_paddr = cookie->dmac_laddress + coffset;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4161 (*windowp)->wd_trim.tr_first_size = trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4162 if (cookie->dmac_type & ROOTNEX_USES_COPYBUF) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4163 (*windowp)->wd_dosync = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4164 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4165
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4166 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4167 * now go back to the current cookie and add it to the new window. set
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4168 * the new window size to the what was left over from the previous
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4169 * cookie and what's in the current cookie.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4170 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4171 cookie++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4172 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4173 (*windowp)->wd_size = trim_sz + cookie->dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4174
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4175 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4176 * trim plus the next cookie could put us over maxxfer (a cookie can be
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4177 * a max size of maxxfer). Handle that case.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4178 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4179 if ((*windowp)->wd_size > dma->dp_maxxfer) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4180 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4181 * maxxfer is already a whole multiple of granularity, and this
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4182 * trim will be <= the previous trim (since a cookie can't be
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4183 * larger than maxxfer). Make things simple here.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4184 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4185 trim_sz = (*windowp)->wd_size - dma->dp_maxxfer;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4186 (*windowp)->wd_trim.tr_trim_last = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4187 (*windowp)->wd_trim.tr_last_cookie = cookie;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4188 (*windowp)->wd_trim.tr_last_paddr = cookie->dmac_laddress;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4189 (*windowp)->wd_trim.tr_last_size = cookie->dmac_size - trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4190 (*windowp)->wd_size -= trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4191 ASSERT((*windowp)->wd_size == dma->dp_maxxfer);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4192
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4193 /* save the buffer offsets for the next window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4194 coffset = cookie->dmac_size - trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4195 new_offset = (windowp)->wd_offset + (windowp)->wd_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4196
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4197 /* setup the next window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4198 (*windowp)++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4199 rootnex_init_win(hp, dma, *windowp, cookie, new_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4200 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4201 (*windowp)->wd_trim.tr_trim_first = B_TRUE;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4202 (*windowp)->wd_trim.tr_first_paddr = cookie->dmac_laddress +
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4203 coffset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4204 (*windowp)->wd_trim.tr_first_size = trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4205 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4206
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4207 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4208 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4209
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4210
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4211 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4212 * rootnex_copybuf_window_boundary()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4213 * Called in bind slowpath when we get to a window boundary because we used
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4214 * up all the copy buffer that we have.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4215 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4216 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4217 rootnex_copybuf_window_boundary(ddi_dma_impl_t hp, rootnex_dma_t dma,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4218 rootnex_window_t *windowp, ddi_dma_cookie_t cookie, off_t cur_offset,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4219 size_t *copybuf_used)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4220 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4221 rootnex_sglinfo_t *sinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4222 off_t new_offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4223 size_t trim_sz;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4224 paddr_t paddr;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4225 off_t coffset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4226 uint_t pidx;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4227 off_t poff;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4228
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4229
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4230 sinfo = &dma->dp_sglinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4231
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4232 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4233 * the copy buffer should be a whole multiple of page size. We know that
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4234 * this cookie is <= MMU_PAGESIZE.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4235 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4236 ASSERT(cookie->dmac_size <= MMU_PAGESIZE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4237
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4238 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4239 * from now on, all new windows in this bind need to be re-mapped during
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4240 * ddi_dma_getwin() (32-bit kernel only). i.e. we ran out out copybuf
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4241 * space...
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4242 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4243 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4244 dma->dp_cb_remaping = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4245 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4246
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4247 /* reset copybuf used */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4248 *copybuf_used = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4249
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4250 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4251 * if we don't have to trim (since granularity is set to 1), go to the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4252 * next window and add the current cookie to it. We know the current
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4253 * cookie uses the copy buffer since we're in this code path.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4254 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4255 if (!dma->dp_trim_required) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4256 (*windowp)++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4257 rootnex_init_win(hp, dma, *windowp, cookie, cur_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4258
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4259 /* Add this cookie to the new window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4260 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4261 (*windowp)->wd_size += cookie->dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4262 *copybuf_used += MMU_PAGESIZE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4263 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4264 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4265
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4266 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4267 * *** may need to trim, figure it out.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4268 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4269
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4270 /* figure out how much we need to trim from the window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4271 if (dma->dp_granularity_power_2) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4272 trim_sz = (*windowp)->wd_size &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4273 (hp->dmai_attr.dma_attr_granular - 1);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4274 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4275 trim_sz = (*windowp)->wd_size % hp->dmai_attr.dma_attr_granular;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4276 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4277
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4278 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4279 * if the window's a whole multiple of granularity, go to the next
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4280 * window, init it, then add in the current cookie. We know the current
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4281 * cookie uses the copy buffer since we're in this code path.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4282 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4283 if (trim_sz == 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4284 (*windowp)++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4285 rootnex_init_win(hp, dma, *windowp, cookie, cur_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4286
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4287 /* Add this cookie to the new window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4288 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4289 (*windowp)->wd_size += cookie->dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4290 *copybuf_used += MMU_PAGESIZE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4291 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4292 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4293
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4294 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4295 * *** We figured it out, we definitly need to trim
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4296 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4297
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4298 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4299 * make sure the driver isn't making us do something bad...
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4300 * Trimming and sgllen == 1 don't go together.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4301 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4302 if (hp->dmai_attr.dma_attr_sgllen == 1) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4303 return (DDI_DMA_NOMAPPING);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4304 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4305
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4306 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4307 * first, setup the current window to account for the trim. Need to go
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4308 * back to the last cookie for this. Some of the last cookie will be in
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4309 * the current window, and some of the last cookie will be in the new
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4310 * window. All of the current cookie will be in the new window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4311 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4312 cookie--;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4313 (*windowp)->wd_trim.tr_trim_last = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4314 (*windowp)->wd_trim.tr_last_cookie = cookie;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4315 (*windowp)->wd_trim.tr_last_paddr = cookie->dmac_laddress;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4316 ASSERT(cookie->dmac_size > trim_sz);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4317 (*windowp)->wd_trim.tr_last_size = cookie->dmac_size - trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4318 (*windowp)->wd_size -= trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4319
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4320 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4321 * we're trimming the last cookie (not the current cookie). So that
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4322 * last cookie may have or may not have been using the copy buffer (
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4323 * we know the cookie passed in uses the copy buffer since we're in
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4324 * this code path).
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4325 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4326 * If the last cookie doesn't use the copy buffer, nothing special to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4327 * do. However, if it does uses the copy buffer, it will be both the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4328 * last page in the current window and the first page in the next
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4329 * window. Since we are reusing the copy buffer (and KVA space on the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4330 * 32-bit kernel), this page will use the end of the copy buffer in the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4331 * current window, and the start of the copy buffer in the next window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4332 * Track that info... The cookie physical address was already set to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4333 * the copy buffer physical address in setup_cookie..
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4334 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4335 if (cookie->dmac_type & ROOTNEX_USES_COPYBUF) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4336 pidx = (sinfo->si_buf_offset + (*windowp)->wd_offset +
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4337 (*windowp)->wd_size) >> MMU_PAGESHIFT;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4338 (*windowp)->wd_trim.tr_last_copybuf_win = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4339 (*windowp)->wd_trim.tr_last_pidx = pidx;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4340 (*windowp)->wd_trim.tr_last_cbaddr =
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4341 dma->dp_pgmap[pidx].pm_cbaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4342 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4343 (*windowp)->wd_trim.tr_last_kaddr =
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4344 dma->dp_pgmap[pidx].pm_kaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4345 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4346 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4347
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4348 /* save the buffer offsets for the next window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4349 coffset = cookie->dmac_size - trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4350 new_offset = (windowp)->wd_offset + (windowp)->wd_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4351
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4352 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4353 * set this now in case this is the first window. all other cases are
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4354 * set in dma_win()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4355 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4356 cookie->dmac_size = (*windowp)->wd_trim.tr_last_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4357
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4358 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4359 * initialize the next window using what's left over in the previous
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4360 * cookie.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4361 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4362 (*windowp)++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4363 rootnex_init_win(hp, dma, *windowp, cookie, new_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4364 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4365 (*windowp)->wd_trim.tr_trim_first = B_TRUE;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4366 (*windowp)->wd_trim.tr_first_paddr = cookie->dmac_laddress + coffset;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4367 (*windowp)->wd_trim.tr_first_size = trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4368
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4369 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4370 * again, we're tracking if the last cookie uses the copy buffer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4371 * read the comment above for more info on why we need to track
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4372 * additional state.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4373 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4374 * For the first cookie in the new window, we need reset the physical
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4375 * address to DMA into to the start of the copy buffer plus any
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4376 * initial page offset which may be present.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4377 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4378 if (cookie->dmac_type & ROOTNEX_USES_COPYBUF) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4379 (*windowp)->wd_dosync = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4380 (*windowp)->wd_trim.tr_first_copybuf_win = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4381 (*windowp)->wd_trim.tr_first_pidx = pidx;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4382 (*windowp)->wd_trim.tr_first_cbaddr = dma->dp_cbaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4383 poff = (*windowp)->wd_trim.tr_first_paddr & MMU_PAGEOFFSET;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4384
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4385 paddr = pfn_to_pa(hat_getpfnum(kas.a_hat, dma->dp_cbaddr)) +
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4386 poff;
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4387 (*windowp)->wd_trim.tr_first_paddr =
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4388 ROOTNEX_PADDR_TO_RBASE(paddr);
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4389
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4390 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4391 (*windowp)->wd_trim.tr_first_kaddr = dma->dp_kva;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4392 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4393 /* account for the cookie copybuf usage in the new window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4394 *copybuf_used += MMU_PAGESIZE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4395
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4396 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4397 * every piece of code has to have a hack, and here is this
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4398 * ones :-)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4399 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4400 * There is a complex interaction between setup_cookie and the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4401 * copybuf window boundary. The complexity had to be in either
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4402 * the maxxfer window, or the copybuf window, and I chose the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4403 * copybuf code.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4404 *
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4405 * So in this code path, we have taken the last cookie,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4406 * virtually broken it in half due to the trim, and it happens
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4407 * to use the copybuf which further complicates life. At the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4408 * same time, we have already setup the current cookie, which
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4409 * is now wrong. More background info: the current cookie uses
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4410 * the copybuf, so it is only a page long max. So we need to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4411 * fix the current cookies copy buffer address, physical
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4412 * address, and kva for the 32-bit kernel. We due this by
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4413 * bumping them by page size (of course, we can't due this on
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4414 * the physical address since the copy buffer may not be
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4415 * physically contiguous).
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4416 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4417 cookie++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4418 dma->dp_pgmap[pidx + 1].pm_cbaddr += MMU_PAGESIZE;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4419 poff = cookie->dmac_laddress & MMU_PAGEOFFSET;
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4420
7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4421 paddr = pfn_to_pa(hat_getpfnum(kas.a_hat,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4422 dma->dp_pgmap[pidx + 1].pm_cbaddr)) + poff;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4423 cookie->dmac_laddress = ROOTNEX_PADDR_TO_RBASE(paddr);
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4424
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4425 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4426 ASSERT(dma->dp_pgmap[pidx + 1].pm_mapped == B_FALSE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4427 dma->dp_pgmap[pidx + 1].pm_kaddr += MMU_PAGESIZE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4428 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4429 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4430 /* go back to the current cookie */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4431 cookie++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4432 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4433
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4434 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4435 * add the current cookie to the new window. set the new window size to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4436 * the what was left over from the previous cookie and what's in the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4437 * current cookie.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4438 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4439 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4440 (*windowp)->wd_size = trim_sz + cookie->dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4441 ASSERT((*windowp)->wd_size < dma->dp_maxxfer);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4442
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4443 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4444 * we know that the cookie passed in always uses the copy buffer. We
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4445 * wouldn't be here if it didn't.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4446 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4447 *copybuf_used += MMU_PAGESIZE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4448
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4449 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4450 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4451
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4452
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4453 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4454 * rootnex_maxxfer_window_boundary()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4455 * Called in bind slowpath when we get to a window boundary because we will
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4456 * go over maxxfer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4457 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4458 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4459 rootnex_maxxfer_window_boundary(ddi_dma_impl_t hp, rootnex_dma_t dma,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4460 rootnex_window_t *windowp, ddi_dma_cookie_t cookie)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4461 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4462 size_t dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4463 off_t new_offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4464 size_t trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4465 off_t coffset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4466
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4467
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4468 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4469 * calculate how much we have to trim off of the current cookie to equal
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4470 * maxxfer. We don't have to account for granularity here since our
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4471 * maxxfer already takes that into account.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4472 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4473 trim_sz = ((*windowp)->wd_size + cookie->dmac_size) - dma->dp_maxxfer;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4474 ASSERT(trim_sz <= cookie->dmac_size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4475 ASSERT(trim_sz <= dma->dp_maxxfer);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4476
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4477 /* save cookie size since we need it later and we might change it */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4478 dmac_size = cookie->dmac_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4479
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4480 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4481 * if we're not trimming the entire cookie, setup the current window to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4482 * account for the trim.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4483 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4484 if (trim_sz < cookie->dmac_size) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4485 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4486 (*windowp)->wd_trim.tr_trim_last = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4487 (*windowp)->wd_trim.tr_last_cookie = cookie;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4488 (*windowp)->wd_trim.tr_last_paddr = cookie->dmac_laddress;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4489 (*windowp)->wd_trim.tr_last_size = cookie->dmac_size - trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4490 (*windowp)->wd_size = dma->dp_maxxfer;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4491
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4492 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4493 * set the adjusted cookie size now in case this is the first
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4494 * window. All other windows are taken care of in get win
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4495 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4496 cookie->dmac_size = (*windowp)->wd_trim.tr_last_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4497 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4498
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4499 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4500 * coffset is the current offset within the cookie, new_offset is the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4501 * current offset with the entire buffer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4502 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4503 coffset = dmac_size - trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4504 new_offset = (windowp)->wd_offset + (windowp)->wd_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4505
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4506 /* initialize the next window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4507 (*windowp)++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4508 rootnex_init_win(hp, dma, *windowp, cookie, new_offset);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4509 (*windowp)->wd_cookie_cnt++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4510 (*windowp)->wd_size = trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4511 if (trim_sz < dmac_size) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4512 (*windowp)->wd_trim.tr_trim_first = B_TRUE;
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4513 (*windowp)->wd_trim.tr_first_paddr = cookie->dmac_laddress +
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4514 coffset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4515 (*windowp)->wd_trim.tr_first_size = trim_sz;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4516 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4517
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4518 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4519 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4520
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4521
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4522 /ARGSUSED/
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4523 static int
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4524 rootnex_coredma_sync(dev_info_t dip, dev_info_t rdip, ddi_dma_handle_t handle,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4525 off_t off, size_t len, uint_t cache_flags)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4526 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4527 rootnex_sglinfo_t *sinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4528 rootnex_pgmap_t *cbpage;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4529 rootnex_window_t *win;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4530 ddi_dma_impl_t *hp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4531 rootnex_dma_t *dma;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4532 caddr_t fromaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4533 caddr_t toaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4534 uint_t psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4535 off_t offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4536 uint_t pidx;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4537 size_t size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4538 off_t poff;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4539 int e;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4540
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4541
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4542 hp = (ddi_dma_impl_t *)handle;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4543 dma = (rootnex_dma_t *)hp->dmai_private;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4544 sinfo = &dma->dp_sglinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4545
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4546 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4547 * if we don't have any windows, we don't need to sync. A copybuf
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4548 * will cause us to have at least one window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4549 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4550 if (dma->dp_window == NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4551 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4552 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4553
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4554 /* This window may not need to be sync'd */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4555 win = &dma->dp_window[dma->dp_current_win];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4556 if (!win->wd_dosync) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4557 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4558 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4559
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4560 /* handle off and len special cases */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4561 if ((off == 0) \|\| (rootnex_sync_ignore_params)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4562 offset = win->wd_offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4563 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4564 offset = off;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4565 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4566 if ((len == 0) \|\| (rootnex_sync_ignore_params)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4567 size = win->wd_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4568 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4569 size = len;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4570 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4571
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4572 /* check the sync args to make sure they make a little sense */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4573 if (rootnex_sync_check_parms) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4574 e = rootnex_valid_sync_parms(hp, win, offset, size,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4575 cache_flags);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4576 if (e != DDI_SUCCESS) {
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	4577 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_SYNC_FAIL]);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4578 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4579 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4580 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4581
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4582 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4583 * special case the first page to handle the offset into the page. The
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4584 * offset to the current page for our buffer is the offset into the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4585 * first page of the buffer plus our current offset into the buffer
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4586 * itself, masked of course.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4587 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4588 poff = (sinfo->si_buf_offset + offset) & MMU_PAGEOFFSET;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4589 psize = MIN((MMU_PAGESIZE - poff), size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4590
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4591 /* go through all the pages that we want to sync */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4592 while (size > 0) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4593 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4594 * Calculate the page index relative to the start of the buffer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4595 * The index to the current page for our buffer is the offset
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4596 * into the first page of the buffer plus our current offset
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4597 * into the buffer itself, shifted of course...
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4598 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4599 pidx = (sinfo->si_buf_offset + offset) >> MMU_PAGESHIFT;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4600 ASSERT(pidx < sinfo->si_max_pages);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4601
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4602 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4603 * if this page uses the copy buffer, we need to sync it,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4604 * otherwise, go on to the next page.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4605 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4606 cbpage = &dma->dp_pgmap[pidx];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4607 ASSERT((cbpage->pm_uses_copybuf == B_TRUE) \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4608 (cbpage->pm_uses_copybuf == B_FALSE));
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4609 if (cbpage->pm_uses_copybuf) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4610 /* cbaddr and kaddr should be page aligned */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4611 ASSERT(((uintptr_t)cbpage->pm_cbaddr &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4612 MMU_PAGEOFFSET) == 0);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4613 ASSERT(((uintptr_t)cbpage->pm_kaddr &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4614 MMU_PAGEOFFSET) == 0);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4615
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4616 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4617 * if we're copying for the device, we are going to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4618 * copy from the drivers buffer and to the rootnex
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4619 * allocated copy buffer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4620 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4621 if (cache_flags == DDI_DMA_SYNC_FORDEV) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4622 fromaddr = cbpage->pm_kaddr + poff;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4623 toaddr = cbpage->pm_cbaddr + poff;
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	4624 ROOTNEX_DPROBE2(rootnex__sync__dev,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4625 dev_info_t *, dma->dp_dip, size_t, psize);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4626
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4627 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4628 * if we're copying for the cpu/kernel, we are going to
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4629 * copy from the rootnex allocated copy buffer to the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4630 * drivers buffer.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4631 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4632 } else {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4633 fromaddr = cbpage->pm_cbaddr + poff;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4634 toaddr = cbpage->pm_kaddr + poff;
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	4635 ROOTNEX_DPROBE2(rootnex__sync__cpu,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4636 dev_info_t *, dma->dp_dip, size_t, psize);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4637 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4638
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4639 bcopy(fromaddr, toaddr, psize);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4640 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4641
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4642 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4643 * decrement size until we're done, update our offset into the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4644 * buffer, and get the next page size.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4645 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4646 size -= psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4647 offset += psize;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4648 psize = MIN(MMU_PAGESIZE, size);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4649
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4650 /* page offset is zero for the rest of this loop */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4651 poff = 0;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4652 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4653
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4654 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4655 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4656
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4657 /*
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4658 * rootnex_dma_sync()
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4659 * called from ddi_dma_sync() if DMP_NOSYNC is not set in hp->dmai_rflags.
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4660 * We set DMP_NOSYNC if we're not using the copy buffer. If DMP_NOSYNC
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4661 * is set, ddi_dma_sync() returns immediately passing back success.
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4662 */
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4663 /ARGSUSED/
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4664 static int
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4665 rootnex_dma_sync(dev_info_t dip, dev_info_t rdip, ddi_dma_handle_t handle,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4666 off_t off, size_t len, uint_t cache_flags)
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4667 {
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	4668 #if defined(__amd64) && !defined(__xpv)
10216 6559aefeb57e 6859941 fix for 6855752 is incorrect Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 10007 diff changeset	4669 if (IOMMU_USED(rdip)) {
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4670 return (iommulib_nexdma_sync(dip, rdip, handle, off, len,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4671 cache_flags));
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4672 }
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4673 #endif
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4674 return (rootnex_coredma_sync(dip, rdip, handle, off, len,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4675 cache_flags));
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4676 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4677
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4678 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4679 * rootnex_valid_sync_parms()
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4680 * checks the parameters passed to sync to verify they are correct.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4681 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4682 static int
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4683 rootnex_valid_sync_parms(ddi_dma_impl_t hp, rootnex_window_t win,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4684 off_t offset, size_t size, uint_t cache_flags)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4685 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4686 off_t woffset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4687
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4688
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4689 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4690 * the first part of the test to make sure the offset passed in is
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4691 * within the window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4692 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4693 if (offset < win->wd_offset) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4694 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4695 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4696
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4697 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4698 * second and last part of the test to make sure the offset and length
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4699 * passed in is within the window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4700 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4701 woffset = offset - win->wd_offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4702 if ((woffset + size) > win->wd_size) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4703 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4704 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4705
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4706 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4707 * if we are sync'ing for the device, the DDI_DMA_WRITE flag should
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4708 * be set too.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4709 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4710 if ((cache_flags == DDI_DMA_SYNC_FORDEV) &&
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4711 (hp->dmai_rflags & DDI_DMA_WRITE)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4712 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4713 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4714
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4715 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4716 * at this point, either DDI_DMA_SYNC_FORCPU or DDI_DMA_SYNC_FORKERNEL
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4717 * should be set. Also DDI_DMA_READ should be set in the flags.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4718 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4719 if (((cache_flags == DDI_DMA_SYNC_FORCPU) \|\|
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4720 (cache_flags == DDI_DMA_SYNC_FORKERNEL)) &&
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4721 (hp->dmai_rflags & DDI_DMA_READ)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4722 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4723 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4724
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4725 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4726 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4727
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4728
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4729 /ARGSUSED/
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4730 static int
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4731 rootnex_coredma_win(dev_info_t dip, dev_info_t rdip, ddi_dma_handle_t handle,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4732 uint_t win, off_t offp, size_t lenp, ddi_dma_cookie_t *cookiep,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4733 uint_t *ccountp)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4734 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4735 rootnex_window_t *window;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4736 rootnex_trim_t *trim;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4737 ddi_dma_impl_t *hp;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4738 rootnex_dma_t *dma;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4739 ddi_dma_obj_t *dmao;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4740 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4741 rootnex_sglinfo_t *sinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4742 rootnex_pgmap_t *pmap;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4743 uint_t pidx;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4744 uint_t pcnt;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4745 off_t poff;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4746 int i;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4747 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4748
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4749
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4750 hp = (ddi_dma_impl_t *)handle;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4751 dma = (rootnex_dma_t *)hp->dmai_private;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4752 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4753 sinfo = &dma->dp_sglinfo;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4754 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4755
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4756 /* If we try and get a window which doesn't exist, return failure */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4757 if (win >= hp->dmai_nwin) {
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	4758 ROOTNEX_DPROF_INC(&rootnex_cnt[ROOTNEX_CNT_GETWIN_FAIL]);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4759 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4760 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4761
13268 b749961aba64 612 apparent reversal of dvma vs. dma in rootnex Garrett D'Amore <garrett@nexenta.com> parents: 13216 diff changeset	4762 dmao = dma->dp_dvma_used ? &dma->dp_dvma : &dma->dp_dma;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4763
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4764 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4765 * if we don't have any windows, and they're asking for the first
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4766 * window, setup the cookie pointer to the first cookie in the bind.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4767 * setup our return values, then increment the cookie since we return
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4768 * the first cookie on the stack.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4769 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4770 if (dma->dp_window == NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4771 if (win != 0) {
12849 331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	4772 ROOTNEX_DPROF_INC(
331f69c36b0a 6959249 Atomic increment in ddi_dma_addr_bind_handle limits UDP TX throughput on Callisto Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12683 diff changeset	4773 &rootnex_cnt[ROOTNEX_CNT_GETWIN_FAIL]);
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4774 return (DDI_FAILURE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4775 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4776 hp->dmai_cookie = dma->dp_cookies;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4777 *offp = 0;
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4778 *lenp = dmao->dmao_size;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4779 *ccountp = dma->dp_sglinfo.si_sgl_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4780 *cookiep = hp->dmai_cookie[0];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4781 hp->dmai_cookie++;
19431 5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	4782 hp->dmai_ncookies = *ccountp;
5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	4783 hp->dmai_curcookie = 1;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4784 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4785 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4786
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4787 /* sync the old window before moving on to the new one */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4788 window = &dma->dp_window[dma->dp_current_win];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4789 if ((window->wd_dosync) && (hp->dmai_rflags & DDI_DMA_READ)) {
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	4790 (void) rootnex_coredma_sync(dip, rdip, handle, 0, 0,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4791 DDI_DMA_SYNC_FORCPU);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4792 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4793
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4794 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4795 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4796 * before we move to the next window, if we need to re-map, unmap all
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4797 * the pages in this window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4798 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4799 if (dma->dp_cb_remaping) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4800 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4801 * If we switch to this window again, we'll need to map in
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4802 * on the fly next time.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4803 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4804 window->wd_remap_copybuf = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4805
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4806 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4807 * calculate the page index into the buffer where this window
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4808 * starts, and the number of pages this window takes up.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4809 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4810 pidx = (sinfo->si_buf_offset + window->wd_offset) >>
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4811 MMU_PAGESHIFT;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4812 poff = (sinfo->si_buf_offset + window->wd_offset) &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4813 MMU_PAGEOFFSET;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4814 pcnt = mmu_btopr(window->wd_size + poff);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4815 ASSERT((pidx + pcnt) <= sinfo->si_max_pages);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4816
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4817 /* unmap pages which are currently mapped in this window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4818 for (i = 0; i < pcnt; i++) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4819 if (dma->dp_pgmap[pidx].pm_mapped) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4820 hat_unload(kas.a_hat,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4821 dma->dp_pgmap[pidx].pm_kaddr, MMU_PAGESIZE,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4822 HAT_UNLOAD);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4823 dma->dp_pgmap[pidx].pm_mapped = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4824 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4825 pidx++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4826 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4827 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4828 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4829
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4830 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4831 * Move to the new window.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4832 * NOTE: current_win must be set for sync to work right
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4833 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4834 dma->dp_current_win = win;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4835 window = &dma->dp_window[win];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4836
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4837 /* if needed, adjust the first and/or last cookies for trim */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4838 trim = &window->wd_trim;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4839 if (trim->tr_trim_first) {
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4840 window->wd_first_cookie->dmac_laddress = trim->tr_first_paddr;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4841 window->wd_first_cookie->dmac_size = trim->tr_first_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4842 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4843 window->wd_first_cookie->dmac_type =
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4844 (window->wd_first_cookie->dmac_type &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4845 ROOTNEX_USES_COPYBUF) + window->wd_offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4846 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4847 if (trim->tr_first_copybuf_win) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4848 dma->dp_pgmap[trim->tr_first_pidx].pm_cbaddr =
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4849 trim->tr_first_cbaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4850 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4851 dma->dp_pgmap[trim->tr_first_pidx].pm_kaddr =
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4852 trim->tr_first_kaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4853 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4854 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4855 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4856 if (trim->tr_trim_last) {
5084 7d838c5c0eed PSARC 2006/260 Solaris on Xen johnlev parents: 2580 diff changeset	4857 trim->tr_last_cookie->dmac_laddress = trim->tr_last_paddr;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4858 trim->tr_last_cookie->dmac_size = trim->tr_last_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4859 if (trim->tr_last_copybuf_win) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4860 dma->dp_pgmap[trim->tr_last_pidx].pm_cbaddr =
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4861 trim->tr_last_cbaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4862 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4863 dma->dp_pgmap[trim->tr_last_pidx].pm_kaddr =
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4864 trim->tr_last_kaddr;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4865 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4866 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4867 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4868
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4869 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4870 * setup the cookie pointer to the first cookie in the window. setup
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4871 * our return values, then increment the cookie since we return the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4872 * first cookie on the stack.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4873 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4874 hp->dmai_cookie = window->wd_first_cookie;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4875 *offp = window->wd_offset;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4876 *lenp = window->wd_size;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4877 *ccountp = window->wd_cookie_cnt;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4878 *cookiep = hp->dmai_cookie[0];
19431 5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	4879 hp->dmai_ncookies = *ccountp;
5599be23ae32 12183 Want new IPD 13 DMA Cookie APIs Robert Mustacchi <rm@fingolfin.org> parents: 16883 diff changeset	4880 hp->dmai_curcookie = 1;
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4881 hp->dmai_cookie++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4882
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4883 #if !defined(__amd64)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4884 /* re-map copybuf if required for this window */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4885 if (dma->dp_cb_remaping) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4886 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4887 * calculate the page index into the buffer where this
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4888 * window starts.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4889 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4890 pidx = (sinfo->si_buf_offset + window->wd_offset) >>
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4891 MMU_PAGESHIFT;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4892 ASSERT(pidx < sinfo->si_max_pages);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4893
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4894 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4895 * the first page can get unmapped if it's shared with the
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4896 * previous window. Even if the rest of this window is already
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4897 * mapped in, we need to still check this one.
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4898 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4899 pmap = &dma->dp_pgmap[pidx];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4900 if ((pmap->pm_uses_copybuf) && (pmap->pm_mapped == B_FALSE)) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4901 if (pmap->pm_pp != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4902 pmap->pm_mapped = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4903 i86_pp_map(pmap->pm_pp, pmap->pm_kaddr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4904 } else if (pmap->pm_vaddr != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4905 pmap->pm_mapped = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4906 i86_va_map(pmap->pm_vaddr, sinfo->si_asp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4907 pmap->pm_kaddr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4908 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4909 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4910 pidx++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4911
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4912 /* map in the rest of the pages if required */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4913 if (window->wd_remap_copybuf) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4914 window->wd_remap_copybuf = B_FALSE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4915
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4916 /* figure out many pages this window takes up */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4917 poff = (sinfo->si_buf_offset + window->wd_offset) &
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4918 MMU_PAGEOFFSET;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4919 pcnt = mmu_btopr(window->wd_size + poff);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4920 ASSERT(((pidx - 1) + pcnt) <= sinfo->si_max_pages);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4921
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4922 /* map pages which require it */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4923 for (i = 1; i < pcnt; i++) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4924 pmap = &dma->dp_pgmap[pidx];
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4925 if (pmap->pm_uses_copybuf) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4926 ASSERT(pmap->pm_mapped == B_FALSE);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4927 if (pmap->pm_pp != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4928 pmap->pm_mapped = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4929 i86_pp_map(pmap->pm_pp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4930 pmap->pm_kaddr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4931 } else if (pmap->pm_vaddr != NULL) {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4932 pmap->pm_mapped = B_TRUE;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4933 i86_va_map(pmap->pm_vaddr,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4934 sinfo->si_asp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4935 pmap->pm_kaddr);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4936 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4937 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4938 pidx++;
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4939 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4940 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4941 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4942 #endif
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4943
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4944 /* if the new window uses the copy buffer, sync it for the device */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4945 if ((window->wd_dosync) && (hp->dmai_rflags & DDI_DMA_WRITE)) {
8215 dc6a5d93e70c 6774448 Toonie needs non-unity map Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7656 diff changeset	4946 (void) rootnex_coredma_sync(dip, rdip, handle, 0, 0,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4947 DDI_DMA_SYNC_FORDEV);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4948 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4949
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4950 return (DDI_SUCCESS);
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4951 }
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4952
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4953 /*
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4954 * rootnex_dma_win()
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4955 * called from ddi_dma_getwin()
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4956 */
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4957 /ARGSUSED/
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4958 static int
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4959 rootnex_dma_win(dev_info_t dip, dev_info_t rdip, ddi_dma_handle_t handle,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4960 uint_t win, off_t offp, size_t lenp, ddi_dma_cookie_t *cookiep,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4961 uint_t *ccountp)
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4962 {
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	4963 #if defined(__amd64) && !defined(__xpv)
10216 6559aefeb57e 6859941 fix for 6855752 is incorrect Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 10007 diff changeset	4964 if (IOMMU_USED(rdip)) {
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4965 return (iommulib_nexdma_win(dip, rdip, handle, win, offp, lenp,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4966 cookiep, ccountp));
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4967 }
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4968 #endif
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4969
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4970 return (rootnex_coredma_win(dip, rdip, handle, win, offp, lenp,
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4971 cookiep, ccountp));
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	4972 }
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	4973
13065 9377d65d657e 6955196 Intel IOMMU code should use higher-level abstraction interface (fix lint) Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 13062 diff changeset	4974 #if defined(__amd64) && !defined(__xpv)
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4975 /ARGSUSED/
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4976 static int
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4977 rootnex_coredma_hdl_setprivate(dev_info_t dip, dev_info_t rdip,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4978 ddi_dma_handle_t handle, void *v)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4979 {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4980 ddi_dma_impl_t *hp;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4981 rootnex_dma_t *dma;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4982
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4983 hp = (ddi_dma_impl_t *)handle;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4984 dma = (rootnex_dma_t *)hp->dmai_private;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4985 dma->dp_iommu_private = v;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4986
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4987 return (DDI_SUCCESS);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4988 }
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4989
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4990 /ARGSUSED/
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4991 static void *
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4992 rootnex_coredma_hdl_getprivate(dev_info_t dip, dev_info_t rdip,
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4993 ddi_dma_handle_t handle)
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4994 {
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4995 ddi_dma_impl_t *hp;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4996 rootnex_dma_t *dma;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4997
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4998 hp = (ddi_dma_impl_t *)handle;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	4999 dma = (rootnex_dma_t *)hp->dmai_private;
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	5000
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	5001 return (dma->dp_iommu_private);
515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	5002 }
13065 9377d65d657e 6955196 Intel IOMMU code should use higher-level abstraction interface (fix lint) Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 13062 diff changeset	5003 #endif
13062 515b1e9bea30 6955192 Intel IOMMU code performs unnecessary write buffer flushes Frank Van Der Linden <frank.van.der.linden@oracle.com> parents: 12849 diff changeset	5004
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5005 /*
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5006 * ************************
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5007 * obsoleted dma routines
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5008 * ************************
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5009 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5010
10216 6559aefeb57e 6859941 fix for 6855752 is incorrect Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 10007 diff changeset	5011 /*
6559aefeb57e 6859941 fix for 6855752 is incorrect Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 10007 diff changeset	5012 * rootnex_dma_mctl()
6559aefeb57e 6859941 fix for 6855752 is incorrect Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 10007 diff changeset	5013 *
13636 776c3d9cae69 998 obsolete DMA driver interfaces should be removed Garrett D'Amore <garrett@damore.org> parents: 13631 diff changeset	5014 * We don't support this legacy interface any more on x86.
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	5015 */
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	5016 /* ARGSUSED */
e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	5017 static int
10216 6559aefeb57e 6859941 fix for 6855752 is incorrect Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 10007 diff changeset	5018 rootnex_dma_mctl(dev_info_t dip, dev_info_t rdip, ddi_dma_handle_t handle,
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5019 enum ddi_dma_ctlops request, off_t offp, size_t lenp, caddr_t *objpp,
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5020 uint_t cache_flags)
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5021 {
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5022 /*
13636 776c3d9cae69 998 obsolete DMA driver interfaces should be removed Garrett D'Amore <garrett@damore.org> parents: 13631 diff changeset	5023 * The only thing dma_mctl is usef for anymore is legacy SPARC
776c3d9cae69 998 obsolete DMA driver interfaces should be removed Garrett D'Amore <garrett@damore.org> parents: 13631 diff changeset	5024 * dvma and sbus-specific routines.
509 ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5025 */
ae7fa29cdf98 4699148 some ddivs_dmae assertions FAIL due to ddi_dma* (9f,s) product or manpages bugs mrj parents: 0 diff changeset	5026 return (DDI_FAILURE);
0 68f95e015346 OpenSolaris Launch stevel@tonic-gate parents: diff changeset	5027 }
1414 b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	5028
7613 e49de7ec7617 PSARC 2008/561 AMD IOMMU Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 7589 diff changeset	5029 /*
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5030 * *********
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5031 * FMA Code
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5032 * *********
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5033 */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5034
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5035 /*
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5036 * rootnex_fm_init()
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5037 * FMA init busop
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5038 */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5039 /* ARGSUSED */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5040 static int
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5041 rootnex_fm_init(dev_info_t dip, dev_info_t tdip, int tcap,
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5042 ddi_iblock_cookie_t *ibc)
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5043 {
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5044 *ibc = rootnex_state->r_err_ibc;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5045
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5046 return (ddi_system_fmcap);
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5047 }
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5048
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5049 /*
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5050 * rootnex_dma_check()
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5051 * Function called after a dma fault occurred to find out whether the
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5052 * fault address is associated with a driver that is able to handle faults
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5053 * and recover from faults.
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5054 */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5055 /* ARGSUSED */
1414 b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	5056 static int
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5057 rootnex_dma_check(dev_info_t dip, const void handle, const void *addr,
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5058 const void *not_used)
1414 b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	5059 {
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5060 rootnex_window_t *window;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5061 uint64_t start_addr;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5062 uint64_t fault_addr;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5063 ddi_dma_impl_t *hp;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5064 rootnex_dma_t *dma;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5065 uint64_t end_addr;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5066 size_t csize;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5067 int i;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5068 int j;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5069
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5070
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5071 /* The driver has to set DDI_DMA_FLAGERR to recover from dma faults */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5072 hp = (ddi_dma_impl_t *)handle;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5073 ASSERT(hp);
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5074
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5075 dma = (rootnex_dma_t *)hp->dmai_private;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5076
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5077 /* Get the address that we need to search for */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5078 fault_addr = (uint64_t )addr;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5079
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5080 /*
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5081 * if we don't have any windows, we can just walk through all the
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5082 * cookies.
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5083 */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5084 if (dma->dp_window == NULL) {
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5085 /* for each cookie */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5086 for (i = 0; i < dma->dp_sglinfo.si_sgl_size; i++) {
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5087 /*
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5088 * if the faulted address is within the physical address
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5089 * range of the cookie, return DDI_FM_NONFATAL.
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5090 */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5091 if ((fault_addr >= dma->dp_cookies[i].dmac_laddress) &&
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5092 (fault_addr <= (dma->dp_cookies[i].dmac_laddress +
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5093 dma->dp_cookies[i].dmac_size))) {
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5094 return (DDI_FM_NONFATAL);
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5095 }
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5096 }
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5097
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5098 /* fault_addr not within this DMA handle */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5099 return (DDI_FM_UNKNOWN);
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5100 }
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5101
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5102 /* we have mutiple windows, walk through each window */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5103 for (i = 0; i < hp->dmai_nwin; i++) {
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5104 window = &dma->dp_window[i];
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5105
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5106 /* Go through all the cookies in the window */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5107 for (j = 0; j < window->wd_cookie_cnt; j++) {
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5108
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5109 start_addr = window->wd_first_cookie[j].dmac_laddress;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5110 csize = window->wd_first_cookie[j].dmac_size;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5111
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5112 /*
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5113 * if we are trimming the first cookie in the window,
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5114 * and this is the first cookie, adjust the start
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5115 * address and size of the cookie to account for the
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5116 * trim.
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5117 */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5118 if (window->wd_trim.tr_trim_first && (j == 0)) {
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5119 start_addr = window->wd_trim.tr_first_paddr;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5120 csize = window->wd_trim.tr_first_size;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5121 }
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5122
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5123 /*
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5124 * if we are trimming the last cookie in the window,
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5125 * and this is the last cookie, adjust the start
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5126 * address and size of the cookie to account for the
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5127 * trim.
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5128 */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5129 if (window->wd_trim.tr_trim_last &&
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5130 (j == (window->wd_cookie_cnt - 1))) {
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5131 start_addr = window->wd_trim.tr_last_paddr;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5132 csize = window->wd_trim.tr_last_size;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5133 }
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5134
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5135 end_addr = start_addr + csize;
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5136
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5137 /*
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5138 * if the faulted address is within the physical
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5139 * address of the cookie, return DDI_FM_NONFATAL.
1865 c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5140 */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5141 if ((fault_addr >= start_addr) &&
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5142 (fault_addr <= end_addr)) {
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5143 return (DDI_FM_NONFATAL);
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5144 }
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5145 }
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5146 }
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5147
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5148 /* fault_addr not within this DMA handle */
c8b524cdb631 PSARC/2006/217 Access and Dma Handle Error Clear Functions dilpreet parents: 1414 diff changeset	5149 return (DDI_FM_UNKNOWN);
1414 b4126407ac5b PSARC 2006/020 FMA for Athlon 64 and Opteron Processors cindi parents: 1242 diff changeset	5150 }
11600 651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5151
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5152 /ARGSUSED/
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5153 static int
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5154 rootnex_quiesce(dev_info_t *dip)
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5155 {
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5156 #if defined(__amd64) && !defined(__xpv)
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5157 return (immu_quiesce());
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5158 #else
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5159 return (DDI_SUCCESS);
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5160 #endif
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5161 }
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5162
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5163 #if defined(__xpv)
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5164 void
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5165 immu_init(void)
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5166 {
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5167 ;
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5168 }
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5169
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5170 void
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5171 immu_startup(void)
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5172 {
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5173 ;
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5174 }
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5175 /ARGSUSED/
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5176 void
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5177 immu_physmem_update(uint64_t addr, uint64_t size)
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5178 {
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5179 ;
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5180 }
651a9a4f7b5f 6875273 Intel IOMMU needs a rewrite Vikram Hegde <Vikram.Hegde@Sun.COM> parents: 11465 diff changeset	5181 #endif

Mercurial > illumos > git > illumos-gate

annotate usr/src/uts/i86pc/io/rootnex.c @ 19431:5599be23ae32